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Abstract 


The  revised  Officer  Evaluation  System  (OES)  is  designed  to 
provide  accurate  performance  feedback  to  subordinates,  mission- 
oriented  performance  appraisals,  and  decentralized  promotion 
recommendations.  This  study  was  designed  to  gauge  officer 
acceptance  of  the  new  system. 

The  major  objectives  of  this  research  were  to  1)  determine  if  the 
OES  is  perceived  by  officers  to  be  better  than  the  previous  Officer 
Effectiveness  Report  (OER)  system  in  identifying  and  selecting  the  best 
qualified  officers  for  promotion,  2)  find  out  if  ratees  were  receiving 
constructive  feedback  in  accordance  with  Air  Force  guidelines,  and  3) 
determine  if  there  were  significant  differences  in  perceptions  of  key 
subgroups  within  the  sample  population. 

To  compare  the  OES  and  OER  system,  data  were  analyzed  from  a 
1987  study  of  the  OER  system  and  from  the  current  study.  In  addition, 
responses  in  the  current  study  compared  the  two  systems.  To 
evaluate  feedback,  ratees  were  asked  specific  questions  about  their 
latest  formal  feedback  session.  Responses  from  members  of  key 
subgroups  were  analyzed  to  ascertain  the  existence  of  important 
response-group  differences. 

The  results  indicate  that  officers  favor  the  new  OES  over  the  old 
OER  system.  They  believe  the  OES  13  better  at  identifying  varying 
levels  of  job  performance  resulting  in  the  selection  of  the  best  qualified 
for  promotion.  However,  they  did  not  believe  they  had  an  equal  chance 
of  receiving  a  "Definitely  Promote"  recommendation  regardless  of  the 
organization  to  winch  they  were  assigned.  A  majority  of  officers 
agreed  they  were  receiving  feedback  in  accordance  with 
organizational  policy.  No  evidence  of  sex  or  racial  bias  was  found. 
Company  grade  officers  felt  more  favorable  toward  formal  feedback 
and  decentralized  promotion  recommendations  than  field  grade 
officers.  Nonrated  officers  favored  the  new  OES  more  than  rated 
officers  and  felt  more  strongly  about  its  ability  to  identify  and  select  the 
best  qualified. 
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USAF  LINE  OFFICER  PERCEPTIONS 
OF  THE  OFFICER  EVALUATION  SYSTEM 

I.  Introduction 

The  Air  Force  has  just  completed  its  seventh  revision  of  the 
Officer  Evaluation  System  (OES),  effective  1  August  1988.  For  an  Air 
Force  officer  this  represents  a  shift  in  the  process  by  which  officers 
are  selected  for  promotion.  Now  officers  will  be  evaluated  based  on 
their  job  performance,  what  they  have  accomplished,  instead  of  their 
performance  potential,  what  they  may  be  capable  of  accomplishing. 

The  idea  is  simple,  get  back  to  basics.  Stress  job  performance 
above  all  else.  Let  the  officer  know  periodically  how  well  he  or  she  is 
doing,  evaluate  the  officer  s  performance,  and  when  it's  time,  send  a 
clear  message  to  the  promotion  board  whether  or  not  he  or  she 
should  be  promoted. 


Background 

The  Officer  Evaluation  System  is  an  integral  part  of  the  Officer 
Professional  Development  (OPD)  Program.  As  outlined  in  AF 
Pamphlet  36-30,  the  three  objectives  of  the  OPD  program  are: 
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First,  professional  development  must  increase  the  officer's 
qualification  and  ability  to  perform  his  or  her  duties,  now  and  in 
the  near  term.  Formal  training  and  discussions  with 
supervisors  provide  a  foundation  for  building  this  competence. 
However,  the  most  important  contributor  is  likely  to  be  the 
officer's  experiences  in  day-to-day  duties.  Second,  professional 
development  involves  preparing  officers  for  future  leadership 
challenges.  Professional  Military  Education  (PME)  and  most 
other  education  assists  this  effort,  but,  again,  the  key  to  growth 
in  leadership  and  professionalism  is  experience  in  appropriate 
leadership  positions.  The  third  objective  of  professional 
development,  simply  stated,  is  to  ensure  the  people  who  are  best 
qualified  are  advanced  in  grade  and  responsibility.  This  is 
where  the  officer  evaluation  system  fits  in.  (Department  of  the 
Air  Force,  1988b,  p.  2) 

Ensuring  that  the  people  who  are  "best  qualified  are  advanced  in 
grade  and  responsibility”  involves  more  than  just  accurate 
appraisals.  As  explained  by  General  Larry  D  Welch,  Chief  of  Staff  of 
the  Air  Force,  there  are  three  intended  purposes  of  the  OES: 

These  purposes  are,  first,  to  provide  meaningful  feedback  on 
how  the  officer  is  measuring  up  to  expectations  and  advise  on 
how  to  better  meet  growth  goals  that  go  with  those  expectations. 
Secondly,  to  provide  a  reliable,  long-term  cummulative  record  of 
the  officer's  performance  and  the  officer's  potential  based  upon 
that  performance.  And  finally,  to  provide  the  promotion  board 
with  the  soundest  possible  advice  on  the  officer's  promotion 
potential,  again,  based  on  the  officer's  performance. 

(Department  of  the  Air  Force,  1988a,  p.  1) 

One  of  the  difficulties  of  the  previous  Officer  Evaluation  System 

was  that  a  single  form,  the  Officer  Effectiveness  Report  (OER),  was 

used  to  accomplish  all  three  purposes  (Department  of  the  Air  Force, 

1988b).  This  created  a  dilemma  for  the  supervisor,  who  in  trying  to  be 

completely  candid  in  documenting  hi3  subordinate’s  performance  on 

paper,  ran  the  risk  of  hurting  the  subordinate's  chances  for 
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promotion  simply  because  his  "word  pictures"  might  not  be 
competitive  with  other  officers  in  the  same  grade. 

Word  pictures  were  not  the  only  aspects  of  the  OER  that  were 
inflated.  Top-block  ratings  for  all  but  a  small  percentage  of  Air  Force 
officers  became  the  norm.  In  order  to  stand  out,  officers  pursued  jobs 
in  which  general  officer  endorsements  were  assured,  often  moving  to 
higher  headquarters  assignments  with  a  minimum  of  field 
experience.  However,  endorsement  levels  also  became  inflated  with 
"more  captains  eligible  for  major  in  the  promotion  zone  receiving 
general  officer  endorsements  than  could  be  selected  for  promotion" 
(Department  of  the  Air  Force,  1988b,  p.  3). 

Rating  inflation  caused  problems  for  both  the  promotion  boards 
and  the  ratee.  The  promotion  boards,  in  an  effort  to  discern  the  "best 
from  the  rest,"  relied  heavily  on  the  writing  skills  of  the  rater  and  on 
secondary  considerations  such  as  professional  military  education 
and  advanced  degree  programs.  As  a  result,  at  the  same  time  junior 
officers  were  trying  to  build  up  depth  of  experience  in  their  particular 
field,  they  were  simultaneously  burdened  with  additional  "square¬ 
filling"  activities  designed  to  keep  them  competitive  for  promotion.  It 
was  common  for  junior  officers  to  be  working  on  both  profesr icnal 
military  education  by  correspondence  and  a  master's  degree  during 
the  first  few  years  of  commissioned  service.  And  at  the  first 
opportunity,  these  officers  "sought  staff  billets  before  their  experience 
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warranted  it"  (Batezel,  1988,  p.  22). 

Junior  officers  were  attempting  to  leap-frog  through  their 
professional  development  path.  Because  of  the  pressures  of  rating 
inflation  and  "square  filling,"  they  were  seeking  breadth  of 
experience  without  first  establishing  depth  of  experience.  "The  new 
OES,  summed  up  by  General  Welch,  seeks  to  put  the  sequence  right 
again  and  restore  common  sense  to  officer  ratings  and  promotions'" 
(Batezel,  1988,  p.  22). 

The  new  OES  is  quite  different  from  its  predecessor,  the  OER. 

The  OES  features  a  three-part  system  designed  to  accomodate  the 
three  purposes  of  the  OES.  They  are  (a)  meaningful  feedback  to  the 
officer,  (b)  a  reliable,  long-term  record  of  performance,  and  (c)  sound 
advise  to  the  promotion  board  on  the  officer  s  promotion  potential. 

To  introduce  the  OES  to  all  Air  Force  officers,  the  Manpower  & 
Personnel  Center  (MPC)  at  Randolph  AFB,  Texas  sent  a  briefing 
team  to  all  Air  Force  bases  in  April  and  May  of  1988.  The  purpose  of 
these  briefings  was  to  introduce  the  new  system  to  all  officers  at  the 
same  time.  For  officers  that  were  unable  to  attend  the  briefings,  MPC 
produced  a  videotape  introduced  by  General  Welch  covering  the  same 
topics  as  the  briefing.  Media  coverage  prior  to  the  briefings  included 
articles  in  the  Air  Force  Times  and  Airman  magazine. 

Subsequently,  the  Air  Force  distributed  two  Air  Force  pamphlets 
dealing  with  the  OES.  Air  Force  Pamphlet  36-30,  OES.  Your  Guide 
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Officer  Professional  Development  Program,  the  reasons  for  the 
revised  OES,  and  the  three  OES  forms.  The  pamphlet  also  answered 
questions  that  the  typical  officer  might  ask  about  the  OES.  This  was 
superceded  several  months  later  by  a  more  comprehensive  guide  to 
the  OES,  Air  Force  Pamphlet  36-6,  USAF  Officer's  Guide  to  the 
Officer  Evaluation  System.  This  more  recent  pamphlet  discusses  the 
mechanics  of  the  three  OES  forms  in  greater  detail,  instructs  raters 
on  the  proper  method  of  conducting  feedback  sessions,  and  discusses 
the  common  problems  when  forming  comments.  These  problems 
include: 

the  tendency  to  be  too  general  rather  than  providing  specific 
behavioral  examples,  making  comments  abstract  rather  than 
concrete,  using  buzzwords  and  superlatives,  or  providing 
comments  only  on  duty  performance  factors  while  ignoring 
officership  factors.  (Department  of  the  Air  Force,  1988c,  p.  12) 

The  Manpower  &  Personnel  Center  also  produced  a  second  video  tape 

which  is  designed  to  demonstrate  the  feedback  concepts  outlined  in 

the  most  recent  pamphlet.  The  combination  of  briefings,  news 

articles,  pamphlets,  and  video  tapes  is  designed  to  disseminate  as 

much  information  about  the  revised  Officer  Evaluation  System  to  as 

many  officers  as  possible. 
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The  first  quantitative  appraisal  system  was  introduced  to  the 
general  psychological  community  by  D.G.  Paterson  in  1922  (Landy  & 
Farr,  1980).  It  was  the  first  use  of  a  graphic  rating  scale  and  was 
characterized  by  two  things:  (a)  The  rater  was  freed  from  qualitative 
judgments,  and  (b)  the  rater  was  able  to  make  as  fine  a 
discrimination  as  desired  (Landy  &  Farr,  1980). 

Behaviorallv  anchored  rating  scales  (BARS).  No  major 
breakthroughs  in  performance  assessment  occurred  until  1963  when 
Smith  and  Kendall  introduced  a  new  type  of  rating  method  that 
became  known  as  behaviorally  anchored  rating  scales  (BARS,  Smith 
&  Kendall,  1963).  BARS  differed  from  traditional  graphic  rating 
scales  because  the  scale  anchors  contained  examples  of  actual 
behavior  rather  than  trait  labels  or  numerical  values  (Landy  &  Farr, 
1980).  Unfortunately,  this  technique  resulted  in  only  marginal 
improvement  over  graphic  scales.  BARS  also  proved  to  be  more 
expensive  to  develop.  Both  graphic  scales  and  BARS  had  individuals 
rated  on  a  continuum  from  unsatisfactory  to  excellent.  Research 
indicated  both  techniques  were  susceptable  to  bias  from  leniency 
error  (Schwab,  Heneman,  &  DeCotiis,1975). 

Forced-choice  rating  systems.  To  combat  leniency,  forced-choice 
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rating  systems  were  developed.  They  require  the  rater  to  choose 
from  among  a  set  of  alternative  descriptors  (normally  four  items)  a 
subset  that  best  characterizes  the  ratee  (Landy  &  Farr,  1980).  A 
rating  score  is  derived  from  the  sum  of  the  relative  weights  of  the 
descriptors  chosen.  This  method  produces  a  slight  decrease  in 
leniency  error,  but  like  the  BARS  system,  the  modest  improvements 
did  not  offset  the  additional  expense  of  instrument  development. 

Forced  distribution.  A  less  expensive  alternative  for  decreasing 
leniency  error  was  the  forced  distribution  method.  Forced 
distribution  asks  the  rater  to  allocate  fixed  proportions  of  his  or  her 
subordinates  to  rating  categories  representing  a  complete  range  of 
performance  levels.  Typically,  these  distributions  are  designed  to 
approximate  a  normal  distribution  with  the  bulk  of  the  employees 
classified  toward  the  center  of  the  distribution  and  diminishing 
proportions  allocated  to  the  extreme  ends  of  the  curve.  There  is 
frequently  resistance  to  this  technique.  Raters  often  object  to  the 
rigidity  of  the  system,  e.g.,  they  must  classify  a  fixed  proportion  of 
their  subordinates  into  the  lowest  rating  category  (Landy,  1989).  This 
may  be  particularly  frustrating  for  the  rater  who  is  pleased  with  all  of 
his  or  her  subordinates  but  is  forced  to  place  a  percentage  of  them  in 
the  "poor"  category  (Saal  &  Knight,  1988).  Even  if  rater  resistance  to 
forced  distribution  approaches  may  be  overcome,  questions  of  their 
validity  across  heterogeneous  groups  remain.  For  example,  average 
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performance  in  an  elite  group  may  not  be  the  same  as  average 
performance  in  a  less  stellar  group. 

Rater  Training 

Changing  the  method  or  format  of  the  appraisal  has  not  been  the 
only  front  in  the  battle  against  leniency  error.  Rater-training 
strategies,  like  rater-error  training  (RET)  and  rater-accuracy 
training  (RAT),  have  been  shown  to  decrease  leniency  and  halo  error 
in  a  number  of  studies  (Bemadin,  1978;  Bemadin  &  Walter,  1977; 
Borman,  1975;  Ivanevich,  1979;  Latham,  Wexley,  &  Pursell,  1979). 
However,  one  of  the  assumptions  in  rater  training  was  that  reducing 
psychometric  errors  would  increase  performance  rating  accuracy, 
an  assumption  that  has  been  questioned  by  more  recent  research 
(Bemadin  &  Pence,  1980;  Bemadin  &  Buckley,  1981;  McIntyre,  Smith, 
&  Hassett,  1984;  Murphy  &  Balzer,  1989).  In  two  of  these  studies,  the 
rater  training  programs  resulted  in  the  learning  of  new  response 
sets,  not  necessarily  more  accurate  ones  (Bemadin  &  Pence,  1980; 
Bemadin  &  Buckley,  1981).  A  meta-analysis  of  10  stucU  ;s  comparing 
rater  error  to  rater  accuracy  revealed  that  the  average  error- 
accuracy  correlation  was  very  near  zero  (Murphy  &  Balzer,  1989).  A 
decrease  in  rater  error  did  not  necessarily  result  in  a  corresponding 
increase  in  rater  accuracy.  In  fact,  the  results  of  this  study  were 
"more  consistent  with  the  hypothesis  that  rater  errors  contribute  to 
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accuracy  than  with  the  hypothesis  that  they  detract  from  accuracy" 
(Murphy  &  Balzer,  1989,  p.  622).  In  light  of  these  findings,  one  could 
argue  that  rater-training  programs  should  reduce  their 
concentration  on  rater-error  training  and  focus  on  training  raters 
not  only  to  rate  fairly  but  to  observe  accurately.  This  approach  should 
lead  to  increased  accuracy  in  the  performance-appraisal  process 
(Bemadin  &  Pence,  1981). 

Performance  Feedback 

Although  "formal  feedback  systems  are  not  often  made  pert  of 
organizational  policy"  (Casdo,  1987,  p.  101),  for  information  about  job 
performance  to  be  effective  in  motivating  employees  there  "must  be  a 
formal  feedback  system,"  and  it  "must  be  incorporated  into  the 
appraisal  system  from  the  very  beginning"  (Landy,  1989, 
p.  162).  Effective  performance  appraisal  systems  often  begin  with  an 
initial  face-to-face  meeting  between  supervisor  and  subordinate  to 
discuss  acceptable  levels  of  job  performance.  The  frequency  of  follow- 
on  feedback  sessions  should  take  into  account  the  level  of  employee 
performance.  For  poor  performers,  coaching  should  be  done  more 
often;  for  others,  the  frequency  of  feedback  sessions  should  not  be  less 
than  one  year  (Cederblom,  1982).  Supervisors  should  be  encouraged 
to  take  notes  during  the  rating  period  so  their  evaluations  will  reflect 
performance  over  the  entire  period,  not  just  from  recent  events 
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(Wehrenberg,  1988).  Employees  should  be  encouraged  to  prepare  for 
the  appraisal  interview.  In  a  study  at  a  large  midwestem  hospital, 
researchers  found  that  employees  who  spent  time  preparing  for  their 
appraisal  interviews  by  reviewing  their  own  performance  over  the 
rating  period  were  more  satisfied  with  the  review  process,  more 
motivated  to  improve  their  performance,  and  more  likely  to  actually 
improve  performance  than  those  employees  that  were  not  prepared 
(Burke,  Weitzel,  &  Weir,  1978).  Raters  should  be  active  listeners  and 
should  encourage  employees  to  participate  in  their  feedback  sessions 
(Casein,  1987).  When  evaluating  an  employee,  supervisors  should 
concentrate  on  judging  job-related  behaviors  not  personality  or 
mannerisms  (Burke  et  al.,  1978).  Raters  should  be  candid  and 
specific  and  pay  particular  attention  to  the  ordor  in  which  the 
feedback  is  given.  To  build  rapport,  the  supervisor  may  begin  with 
positive  feedback  on  minor  issues  before  progressing  to  larger  issues. 
Praising  an  employee  on  minor  issues  will  put  him  or  her  at  ease 
and  lessens  the  psychological  defense  mechanisms  normally 
activated  by  criticism  (Stone,  Gueutal,  &  McIntosh,  1984).  Mutually 
agreeable  goals  for  future  performance  should  be  set.  Participation 
by  the  subordinate  in  setting  difficult  goals  will  lead  to  a  higher  level 
of  acceptance  and  performance  than  setting  easily  achieved  goals 
(Erez,  Earley,  &  Hulin,  1985).  Finally,  a  formal  appraisal  interview 
should: 
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merely  formalize  a  process  that  should  be  occuring 
regularly  anyway.  Periodic  tracking  of  progress  toward  goals 
helps  keep  behavior  on  target,  provides  a  better  understanding  of 
the  reasons  behind  a  given  level  of  performance,  and  enhances 
a  subordinate  s  commitment  to  effective  performance. 

(Casdo,  1987,  p.  104) 

USAF  Officer  Appraisal  Systems  1947-1988 

Military  appraisal  systems  were  not  always  compromised  by 
leniency  error.  In  1813,  Army  Brig  Gen  Lewis  Cass  recorded  his 
descriptions  of  the  officers  of  the  27th  Infantry  Regiment  with 
phrases  like  "a  good  man,  but  no  officer"  and  "a  knave  despised  by 
all"  (Bass  &  Barret,  1972,  p.  241).  Over  the  subsequent  175  years, 
there  has  been  a  shift  toward  greater  leniency  in  the  performance 
ratings  of  all  the  services. 

Since  it  became  a  separate  service  in  1947,  the  Air  Force  has 
conducted  seven  revisions  of  the  Officer  Effectiveness  Report  (OER). 
Throughout,  there  has  been  a  "fundamental  conflict  between 
administrative  need  for  differentiation"  and  at  the  same  time  "an 
institutional  reluctance  to  identify  less  than  outstanding 
performance"  (Syllogistics  Inc.  &  The  Hay  Group,  1988,  p.  1-2). 

From  1947  to  1949  the  Air  Force  used  a  forced-choice  method 
inherited  from  the  Army.  Due  to  the  nature  of  this  system,  raters  did 
not  know  the  results  of  their  ratings.  Rater  objections  led  to  the 
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implementation  of  a  new  evaluation  system  in  1949.  This  system 
asked  the  rater  to  evaluate  certain  weighted  aspects  of  proficiency 
and  performance.  Inflation  and  a  preoccupation  with  the  total  score 
instead  of  the  individual  scores  doomed  this  system  by  1952. 

The  third  OER  system  lasted  for  eight  years,  and  except  for  the 
controlled  OER  system  from  1974-1978,  the  basic  design  was  still  in 
use  until  1988.  This  instrument  featured  six  performance  factors 
with  graduated  standards  on  the  front  of  the  form  and  an  overall 
rating  and  final  endorsement  on  the  back.  In  1960  a  9-4  system  was 
established  with  nine  representing  the  top  performance  score  and 
four  representing  the  top  promotion  potential  rating  (Syllogistics  Inc. 
&  The  Hay  Group,  1988).  Inflation  rendered  this  system  ineffective 
and  resulted  in  the  controlled  OER  system  in  1974. 

The  controlled  OER  was  by  far  the  most  unpopular  OER  system 
the  Air  Force  ever  used  (Syllogistics,  Inc.  &  The  Hay  Group,  1988). 
Under  the  controlled  OER,  22%  of  all  officers  could  receive  the  highest 
rating,  28%  could  receive  the  next  highest  rating,  and  the  remaining 
50%  were  accorded  lower  ratings.  Top  ratings  were  the  exception 
rather  than  the  rule  and  after  four  years  of  ratee  anxiety  the  system 
was  revised  in  1978  (Syllogistics  Inc.  &  The  Hay  Group,  1988). 

For  the  next  ten  years  the  OER  retained  its  scaled  performance 
factors  with  narrative  comments  on  the  front  side  of  the  form  and 
scaled  promotion  potential  with  endorsements  on  the  back  side. 
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Again,  inflation  took  its  toll  with  over  90%  of  all  officers  receiving 
"fire walled"  (i.e.,  all  frontside  ratings  at  the  highest  level) 
performance  ratings  and  98%  receiving  the  highest  promotion 
potential  ratings  (Pontiff,  1987).  For  all  intents  and  purposes,  the 
level  of  endorsement  (i.e.,  rank  of  endorser)  became  the  only  usefiU 
standard  for  measuring  of  an  officer  s  promotion  potential. 

Officer  Appraisal  Systems  Used  Bv  Other  Military  Services 

U.S.  Army  PER 

Like  the  Air  Force,  the  Army  had  problems  with  rating  inflation 
until  they  revised  their  OER  system  in  1979  (see  sample  Army  OER  in 
Appendix  A).  The  Army  OER  system  begins  when  the  rater  and 
ratee  meet  face-to-face  to  develop  a  duty  description  and  set  major 
performance  objectives  to  be  accomplished  during  the  rating  period 
(Syllogistics,  Inc.  &  The  Hay  Group,  1988).  Rater  and  ratee  are 
expected  to  meet  periodically  in  counseling  review  sessions  until  the 
final  evaluation  takes  place.  At  that  time,  the  ratee  records  his  or  her 
own  performance,  the  rater  evaluates  it  and  passes  it  on  to  the  senior 
rater.  In  addition,  ratings  on  professionalism,  performance,  and 
potential  for  promotion  are  passed  on  to  the  senior  rater. 

The  senior  rater  then  independently  rates  the  ratee  using  a 
center  of  mass"  concept  commonly  refered  to  as  the  "stickmen"  to 
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decide  where  the  ratee  fits  in  relation  to  other  officers  of  the  same 
grade  under  the  rater's  command  (see  Part  VII  of  Army  OER  form 
in  Appendix  A).  Once  completed,  this  form  is  sent  to  the  military 
personnel  center  where  the  rater’s  rating  history  for  officers  of  that 
rank  is  recorded  on  the  form  opposite  the  individual's  rating.  The 
purpose  of  this  analysis  is  to  assess  the  ratee  relative  to  other  officers 
of  the  same  grade. 

If  the  officer  receives,  for  example  ,  the  highest  rating,  and  only 
five  percent  of  all  officers  were  rated  at  the  same  level,  then  this 
officer  would  be  viewed  as  having  received  a  substantially  high  rating 
by  anyone  reviewing  his  or  her  record.  On  the  other  hand,  if  the 
officer  receives  the  highest  rating  and  fifty  percent  of  all  officers  were 
rated  at  the  same  level,  then  it  would  be  difficult  for  anyone  reviewing 
this  officer's  record  to  determine  exactly  how  well  the  officer  had  been 
rated. 

To  keep  senior  raters  from  distorting  or  skewing  the  center  of 
mass  distribution  by  not  realistically  distributing  their  evaluations 
across  all  levels  of  the  rating  continuum,  a  rater's  rating  history  is 
tracked  by  the  military  personnel  center.  If  a  rater  is  skewing  his  or 
her  rating  distribution,  he  or  she  receives  an  official  warning  and  is 
given  the  option  of  starting  the  tracking  process  over  at  zero.  Most 
raters  elect  this  option  and  attempt  to  get  in  line  with  policy  on  center 
of  mass  distributions.  To  insure  that  the  raters  comply,  the  rater’s 
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rating  history  is  entered  into  the  rater's  permanent  promotion  file 
and  is  a  factor  when  he  or  she  comes  up  for  promotion.  This  system 
appears  to  effectively  control  rating  inflation  (Syilogistics,  Inc.  &  The 
Hay  Group,  1988). 

U.S.  Navy  JTTREP 

The  current  U.S.  Navy  Fitness  Report  (FlTREP)  has  been  in 
existence  since  1974  (see  sample  Navy  FlTREP  in  Appendix  B).  A 
distinguishing  feature  of  this  appraisal  technique  is  that  there  is  only 
one  evaluator,  normally  the  commander  (Syilogistics  Inc.  &  The  Hay 
Group,  1988).  Before  the  commander  writes  a  FlTREP,  the  ratee  has 
the  opportunity  to  submit  information  about  his  or  her  performance 
during  the  rating  period  which  may  or  may  not  be  included  in  the 
report.  The  ratee  is  evaluated  on  twelve  performance  factors,  six 
personal  traits,  given  an  overall  performance  rating,  and  a 
promotion  recommendation. 

The  Navy  rates  all  officers  of  the  same  grade  at  the  same  time. 
This  gives  the  senior  rater  the  opportunity  to  rank  order  his  or  her 
officers  based  upon  their  mission  contribution.  This  overall  ranking 
is  recorded  just  below  the  ratee’s  individual  ranking  so  that  anyone 
reviewing  the  ratee's  record  will  have  a  better  understanding  of  how 
he  ir  she  faired  relative  to  others  of  the  same  rank. 

The  report  is  then  forwarded  to  the  Navy  Military  Personnel 


Command  without  further  review.  Unlike  the  Army  and  the  Air 
Force,  where  every  member  of  a  promotion  board  views  every  record, 
each  Navy  promotion  panel  member  is  given  a  separate  set  of  records 
to  review.  Each  panel  member  submits  his  or  her  findings  to  the 
other  panelists  who  then  vote  secretly  to  determine  the  selectees. 
Although  rating  inflation  is  common  in  the  Navy,  the  special 
attention  each  ratee  receives  by  the  promotion  board  is  thought  to 
afford  a  more  finely-grained  distinction  between  selectees  and 
nonselectees  (Syllogistics  Inc.  &  The  Hay  Group,  1988). 

U.S.  Marine  Corps  FITREP 

The  U.S.  Marine  Corps  Officer  Fitness  Report  (FITREP)  was 
revised  in  1985  (see  sample  Marine  Corps  FITREP  in  Appendix  C). 
Like  the  Navy,  these  reports  are  submitted  in  batch  form,  with  all 
officers  of  the  same  rank  evaluated  at  the  same  time.  Ratings  are 
made  on  a  six-point  scale  from  unsatisfactory  to  outstanding. 

Officers  are  rated  on  seven  job  performance  standards  and  fourteen 
personal  quality  traits.  Officers  are  then  rated  on  their  "general 
value  to  the  service.'"  This  rating  may  be  compared  to  the  distribution 
of  ratings  for  all  other  officers.  For  officers  rated  outstanding,  the 
senior  rater  must  rank  the  ratee  in  comparison  to  others  rated 
outstanding.  For  instance,  a  rating  of  2  of  5,  would  classify  an  officer 
as  the  second  best  of  five  officers  receiving  an  outstanding  rating. 
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The  U.S.  Coast  Guard's  OER  is  unique  in  that  it  uses  a 
behaviorially  anchored  rating  scale  (BARS)  system  to  rank  its  officers 
(see  sample  Coast  Guard  OER  in  Appendix  D).  In  fact,  the  Coast 
Guard  has  developed  a  BARS  system  for  each  rank. 

The  ratings  are  made  on  a  seven-point  rating  scale  featuring 
four  examples  of  behavior  for  each  dimension.  Officers  are  rated  by 
the  rater  on  five  items  measuring  performance  of  duties,  two 
measuring  interpersonal  relations,  four  measuring  leadership,  and 
three  measuring  communication  skills.  The  rater  then  rates  the 
officer  on  five  personal  quality  traits  and  four  items  measuring  the 
officer  s  ability  to  properly  represent  the  Coast  Guard.  There  is  also 
room  on  the  form  following  each  section  for  rater  comments 

The  senior  rater  then  describes  the  ratee's  demonstrated 
leadership  and  potential  and  provides  an  overall  evaluation.  Again, 
like  the  Navy  and  the  Marines,  ratings  are  done  in  batch  form,  with 
all  officers  of  the  same  grade  rated  at  the  same  time.  After  the  form 
is  submitted  to  the  personnel  office,  the  rating  distribution  for  all 
other  officers  of  the  same  grade  is  added  directly  below  the 
individual's  score. 
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The  Current  USAF  Officer  Evaluation  System  (OES) 


On  the  1st  of  August,  1988  the  Air  Force  officially  replaced  the 
OER  system  with  the  new  Officer  Evaluation  System  (OES).  This 
change  was  initiated  by  General  Larry  D.  Welch,  Air  Force  Chief  of 
Staff,  who  explained, 

By  early  1987,  the  outcry  over  square-filling  had  become 
heavy  and  compelling.  The  maun  complaint:  job  performance 
seemed  less  and  less  the  measure  of  success.  Instead,  early 
completion  of  professional  military  education  and  rating 
endorsement  levels  were  stealing  the  spotlight.  We  need  to 
restore  the  focus  on  job  performance  and  renew  officer  trust  in 
the  evaluation  system.  (Department  of  the  Air  Force,  1988b, 
p.  19) 

To  accomplish  this,  he  commissioned  three  groups  to  explore 
alternatives  to  the  old  system.  From  the  private  sector,  Syllogistics, 
Inc.  &  The  Hay  Group  conducted  an  independent  appraisal  of  the 
current  system  and  made  recommendations  for  improvement. 

Syllogistics,  Inc.  &  the  Hay  Group  interviewed  a  dozen  senior  Air 
Force  personnel  experts  and  conducted  group  discussions  with  nine 
focus  groups  composed  of  six  to  eight  officers  each.  There  was  a 
strong  belief  among  the  officers  interviewed  that,  because  of  the 
screening  processes  an  officer  goes  through,  the  result  is  an  elite 
corps  well  above  the  "average"  population.  Because  of  this  feeling, 
there  was  some  resistance  by  raters  to  single  officers  out  as  below 
average.  However,  there  was  no  resistance  to  identifying  above 
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average  officers  or  officers  whose  performance  was  so  poor  that  they 
should  be  released  from  the  Air  Force.  There  was  a  feeling  among 
junior  officers,  in  particular,  that  individuals  on  promotion  boards 
don't  have  time  for  any  in-depth  examination  of  an  officer’s  records 
and,  therefore,  make  promotion  decisions  based  on  "surface  data" 
only.  Most  officers  interviewed  agreed  that  some  sort  of  control  on 
inflation  was  needed  and  felt  the  wing  level  was  the  most  logical 
place  to  start.  All  officers  agreed  that  training  for  all  personnel 
involved  in  the  evaluation  process  was  necessary  to  prepare  them  for 
substantial  changes  in  the  method  of  appraisal.  Junior  officers 
expressed  the  strongest  desire  to  receive  feedback  on  their 
performance  from  their  immediate  supervisors.  According  to  the 
study  group,  this  situation  was  not  unlike  civilian  industry  where 
young  professionals  fresh  from  college  frequently  expressed  a  desire 
for  an  "open"  atmosphere  of  communication  between  workers  and 
managers. 

Representing  the  Air  Force,  groups  of  students  from  the  Air 
Command  and  Staff  College  and  active  duty  and  retired  senior  Air 
Force  officers  from  Randolph  Air  Force  Base,  the  home  of  the  Air 
Force's  Manpower  &  Personnel  Center,  studied  the  Air  Force 
appraisal  problem.  "Specific  Air  Force  guidance  for  the  project  was 
that  any  alternative  conceptual  design  to  the  OER  should:  1)  focus  on 
the  officer1  s  current  job  performance;  2)  provide  good  differentiation 


19 


among  officers  on  potential  for  promotion  and  for  successfully 
executing  higher  responsibility;  and  3)  provide  some  vehicle  for 
giving  officers  feedback  on  their  performance  to  support  career 
development  and  counseling"  (Syllogistics,  Inc.  &  The  Hay  Group, 
1988,  p.  v). 

All  three  groups  concluded  that  to  meet  the  specific  guidelines, 
more  than  one  form  would  be  needed.  The  result  was  a  system 
involving  three  procedures  using  three  different  forms. 

The  first  element  of  the  OES  is  the  Performance  Feedback 
Worksheet,  a  handwritten  off-the-record  counseling  tool  used  by  the 
rater  to  confidentially  let  the  ratee  know  what  is  expected  of  him  or 
her  (see  sample  Performance  Feedback  Worksheet  in  Appendix  E). 
No  other  copies  are  filed  or  shown  to  other  individuals.  For  company 
grade  officers  these  counseling  sessions  occur  semi-annually;  for 
field  grade  officers  the  frequency  of  sessions  is  at  the  discretion  of  the 
rater  or  the  ratee. 

The  second  component  of  the  OES  is  the  Officer  Performance 
Report  (see  sample  Officer  Performance  Report  in  Appendix  F).  It  is 
much  like  the  previous  OER,  except  that  it  focuses  exclusively  on  job 
performance  and  endorsement  levels  are  now  controlled  according  to 
the  rank  of  the  ratee.  The  front  side  of  the  form  contains  the  ratee's 
job  description,  an  evaluation  of  the  impact  the  officer  has  had  on  the 
unit's  mission,  and  ratings  against  six  performance  standards.  The 
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back  side  of  the  form  contains  a  narrative  assessment  by  the  rater 
and  a  second  rater,  along  with  a  concur  or  nonconcur  block  for  a 
reviewer.  For  lieutenant  through  major,  the  reviewer  is  a  wing 
commander  or  equivalent  (normally  a  colonel);  for  lieutenant 
colonels  and  colonels,  the  reviewer  is  the  first  general  officer  in  the 
ratee's  chain  of  command.  Under  the  OES,  there  is  now  a  final 
"reviewer"  instead  of  final  "endorser." 

On  the  previous  OER,  the  endorsement  level,  or  rank  of  the 
senior  endorser  on  a  subordinate’ s  OER,  was  often  an  indication  of 
how  much  that  particular  officer  contributed  to  the  organization.  For 
example,  a  captain  who  was  an  exceptional  performer  might  receive 
a  two-  or  three-star  general  officer  endorsement  on  his  or  her  OER, 
while  other  captains  in  the  same  unit  might  only  receive  a  colonel's 
endorsement.  Limiting  the  endorsement  level  to  wing  commander  or 
equivalent  for  majors  and  below  places  emphasis  on  depth  of 
experience  in  line  jobs.  Instructions  on  the  form  state  that  the  rater 
will  not  consider  or  comment  on  completion  of  professional  military 
education,  advanced  education,  or  family  activities.  These  types  of 
activities  were  frequently  emphasized  on  the  previous  OER  form. 

The  third  element  of  the  OES  is  the  Promotion  Recommendation 
Form  (see  sample  Promotion  Recommendation  Form  in  Appendix 
G).  It  is  completed  60  days  prior  to  a  promotion  board  meeting.  The 
form  contains  a  promotion  recommendation  made  by  the  senior  rater 
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of  "Definitely  Promote,"  "Promote,"  or  "Do  Not  Promote  This  Board." 
The  number  of  "Definitely  Promote"  allocations  is  determined  by 
manning  levels  and  distributed  evenly  throughout  the  major 
commands.  Based  upon  current  manpower  levels,  65%  of  all  line 
captains  eligible  for  promotion  to  major  may  receive  a  "Definitely 
Promote"  recommendation.  There  are  no  restrictions  on  the  number 
of  "Promote"  or  "Do  Not  Promote  This  Board"  recommendations. 
Virtually  all  captains  receiving  a  "Definitely  Promote" 
recommendation  are  promoted  to  major  while  about  50%  of  those 
captains  given  a  "Promote"  recommendation  will  be  promoted.  The 
Promotion  Recommendation  Form  also  provides  instructions 
directing  the  rater  to  ignore  professional  military  education  or 
advanced  academic  education  in  deciding  on  the  promotion 
recommendation. 

Once  completed,  the  promotion  recommendations  from  the  unit 
are  sent  to  an  evaluation  board  at  a  higher  headquarters  whose  job  is 
to  screen  the  unit  nominees  records  for  accuracy,  award  "Definitely 
Promotes"  to  deserving  officers  in  units  too  small  to  receive  whole 
number  allocations,  and  to  resolve  any  remaining  allocations  due  to 
rounded-off  percentage  points.  Once  this  intermediate  promotion 
board  has  completed  its  selections,  its  recommendations  are 
forwarded  to  the  Air  Force's  central  promotion  board  for  final 
promotion  decisions. 
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Because  of  budget  requirements,  legislative  controls,  and  other 
factors,  the  U.S.  military  is  burdened  with  an  "up  or  out"  system  in 
which  officers  must  either  achieve  promotion  or  leave  the  system 
before  the  completion  of  a  full  career.  This  puts  stress  upon  the 
officers  and  the  systems  designed  to  identify  the  best  qualified  for 
promotion.  The  result  is  that  each  branch  of  the  service  has 
developed  its  own  appraisal  and  evaluation  system  to  meet  its  own 
unique  needs. 

The  Air  Force  Officer  Evaluation  System  is  different  from  all 
other  military  systems  in  that  there  is  no  peer  ranking,  and  it  utilizes 
forced  distribution  techniques  to  control  the  number  of 
recommendations  given  to  officers  for  promotion.  Inflation  control 
for  purposes  of  promotion  is  handled  at  the  wing  level  or  higher 
where  "Definitely  Promotes"  are  awarded  based  upon  established 
quotas.  Inflation  control  for  the  Army  rests  with  the  senior  rater 
whose  own  career  is  at  stake  if  he  or  she  does  not  conform  to 
established  policies.  For  the  maritime  services,  rating  inflation 
control  is  more  indirect.  Because  these  services  perform  their 
ratings  in  batch  form,  some  control  is  maintained. 

The  Army  and  the  Air  Force  require  formal,  documented 
feedback.  With  the  exception  of  the  Coast  Guard,  the  maritime 
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services  do  not.  The  Coast  Guard  includes  a  formal  counseling 
program  for  its  first  two  junior  grades.  Counseling  in  general  is 
encouraged  in  the  Navy  and  the  Coast  Guard  but  not  considered 
important  by  the  Marines. 


The  purpose  of  the  thesis  is  to  gather  data  on  the  perceptions  of 
both  company  and  field  grade  line  officers  with  regard  to  the  current 
Officer  Evaluation  System.  The  measures  in  the  study  provide  data 
on  the  success  of  the  OES  in  bringing  about  productive  change  and  on 
the  level  of  acceptance  of  the  OES  by  the  officer  corps.  Acceptance  is 
critical  to  the  success  of  any  appraisal  system;  the  OES  is  no 
exception. 


Given  that  a  change  was  warranted,  does  the  new  Officer 
Evaluation  System  achieve  its  intended  objectives?  More  specifically, 
is  the  OES  perceived  to  be  a  better  tool  for  evaluating  officer 
performance  than  the  previous  OER  system?  Does  the  new  system 
accomodate  more  valid  and  discriminating  assessment  of  officer  job 
performance  and  make  more  fair  and  accurate  promotion  decisions? 
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Are  raters  providing  constructive  feedback  to  their  subordinates? 


Thesis  Objectives 


Objective  1 

The  main  objective  of  this  study  was  to  determine  if  the  current 
OES  is  perceived  by  officers  to  be  better  than  the  previous  OER  system. 
The  OES  was  designed  to  better  differentiate  officer  performance  for 
the  purpose  of  making  sound  promotion  decisions  and  to  enhance  the 
level  of  communication  between  the  rater  and  the  ratee.  Four 
hypotheses  support  this  objective. 

Hypothesis  la.  Officers  perceive  the  current  OES  to  be 
significantly  better  than  the  previous  OER  system  in  its  ability  to 
differentiate  on  the  basis  of  job  performance  and  identify  the  most 
qualified  for  promotion. 

Hypothesis  lb.  Officers  perceive  promotion  decisions  rendered 
under  the  current  OES  to  be  more  fair  and  just  than  they  were  under 
the  previous  OER  system. 

Hypothesis  lc.  Officers  perceive  job  performance  to  be  more 
fairly  and  accurately  appraised  under  the  current  OES  than  it  was 
under  the  previous  OER  system. 

Hypothesis  Id.  Officers  perceive  feedback  under  the  current  OES 
to  be  clearer  and  more  timely  than  it  was  under  the  OER  system. 
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Objective  2 


Formal  feedback  is  a  major  change  for  the  OES.  To  assist  raters 
in  conducting  formal  feedback,  the  Air  Force  has  provided 
guidelines  in  Air  Force  Pamphlet  36-6,  USAF  Officer's  Guide  to  the 
Officer  Evaluation  System.  The  purpose  of  Objective  2  was  to 
determine  if  the  feedback  officers  are  receiving  from  their  raters  is  in 
accordance  with  the  guidelines  in  the  pamphlet.  Hypothesis  2  states 
that  a  majority  of  officers  agree  that  they  are  receiving  feedback 
within  the  guidelines. 

Objective  3 

Another  major  change  for  the  OES  is  the  new  emphasis  on  the 
senior  rater's  role  in  promotion  decisions.  The  purpose  of  Objective  3 
was  to  determine  if  ratees  have  more  trust  in  their  senior  rater  than 
in  the  central  promotion  board  in  making  promotion  decisions. 
Hypothesis  3  predicted  that  the  majority  of  officers  would  agree  that 
their  senior  rater  is  more  qualified  to  make  promotion  decisions  than 
the  central  promotion  board. 

Objective  4 

Since  promotion  decisions  under  the  OES  are  formulated  at  the 
unit  level,  Objective  4  seeks  to  determine  if  officers  believe  they  have 
an  equal  chance  of  receiving  a  "Definitely  Promote”  recommendation 
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regardless  of  the  level  of  their  unit  of  assignment.  One  possible 
conflict  involves  officers  in  small  units.  Some  small  units  do  not  have 
enough  assigned  officers  to  receive  any  "Definitely  Promote" 
allocations.  These  officers  are  referred  to  promotion  boards  at  the 
next  higher  organizational  level.  The  situation  may  be  perceived  as  a 
disadvantage  for  the  affected  officers  because  the  promotion  board 
members  will  not  have  the  same  level  of  contact  with  these  officers  as 
they  would  have  with  officers  assigned  within  their  own  unit. 
Hypothesis  4  postulated  that  a  majority  of  officers  believe  they  have  an 
equal  chance  of  receiving  a  "Definitely  Promote"  allocation 
regardless  of  the  organizational  level  to  which  they  are  assigned. 

Objective  5 

OES  forms  are  different  from  the  previous  OER  forms.  The  new 
Performance  Feedback  Worksheet  did  not  exist  under  the  previous 
system  and  the  Officer  Performance  Report  is  somewhat  different 
from  the  previous  Officer  Effectiveness  Report.  The  new  Promotion 
Recommendation  Form  was  not  evaluated  because  it  has  had  little 
use  since  the  OES  was  implemented  13  months  ago.  Objective  5  was 
designed  to  determine  if  the  two  new  forms  are  easy  to  complete  and 
if  they  are  perceived  by  officers  to  be  useful  for  their  intended 
purposes.  Two  hypotheses  supported  this  objective. 

Hypothesis  5a.  A  majority  of  officers  agree  that  they  find  the 
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Performance  Feedback  Worksheet  easy  to  complete  and  useful  for  its 
intended  purpose. 

Hvpthesis  5b.  A  majority  of  officers  agree  that  they  find  the 
Officer  Performance  Report  easier  to  complete  and  a  better  tool  for 
documenting  performance  than  the  previous  Officer  Effectiveness 
Report  form. 

Objective  6 

With  the  increase  in  the  number  of  forms  to  complete  under  the 
OES,  it  is  important  to  determine  if  this  is  a  matter  of  some  concern. 
From  a  similar  study  conducted  on  the  OER  system  (Pontiff,  1987), 
the  amount  of  time  filling  out  forms  was  a  concern  of  the  officers 
surveyed.  Hypothesis  6  predicted  that  officers  would  be  neither  more 
concerned  nor  le  s  concerned  with  the  amount  of  time  spent 
completing  the  current  OES  forms  than  they  were  with  the  time 
spent  completing  the  previous  OER  forms. 

Objective  7 

Differences  in  perceptions  of  the  OES  between  subgroups  of  the 
population,  e.g.,  males  versus  females,  may  signify  problems  of 
fairness  in  an  appraisal  system  that  has  been  designed  to  be  objective 
across  all  constituencies.  The  purpose  of  Objective  7  was  to 
determine  if  any  significant  differences  exist  between  subgroups  in 
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the  population.  Three  hypotheses  supported  this  objective. 


Hypothesis  7a.  There  is  no  evidence  of  racial  bias  or  gender  bias 
in  the  OES. 

Hypothesis  7b.  Company  grade  (i.e.,  lieutenants  and  captains) 
and  field  grade  (i.e.,  majors,  lieutenant  colonels,  and  colonels) 
officers  feel  no  differently  about  the  OES. 

Hypothesis  7c.  Nonrated  (i.e.,  officers  with  no  aeronautical 
rating)  and  rated  officers  feel  no  differently  about  the  OES. 
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II.  Method 


Sam  nip  and  Setting 

The  data  from  this  study  were  collected  from  906  active  duty  Air 
Force  line  officers  in  the  rank  of  second  lieutenant  through  colonel. 
The  distribution  of  respondents  included  65  second  lieutenants,  123 
first  lieutenants,  354  captains,  197  majors,  116  lieutenant  colonels, 
and  51  colonels.  The  906  respondents  represented  approximately  1% 
of  the  current  population  of  90,500  line  officers. 

Data  collected  from  981  line  officers  from  a  similar  9tudy,  PER 
Exceptions  of  Field  and  Company  Grade  Line  Officers,  conducted  by 
Major  Glen  Pontiff  in  1987,  were  compared  with  identical  measures 
incorporated  into  the  current  study  to  determine  if  there  were  any 
significant  differences  in  officer  perception*  of  the  two  appraisal  and 
evaluation  systems.  The  distribution  of  respondents  to  the  Pontiff 
study  included  59  second  lieutenants,  108  first  lieutenants,  294 
captains,  245  majors,  196  lieutenant  colonels,  and  79  colonels. 

For  both  studies,  officers  were  selected  randomly  from  the  Air 
Force  s  Atlas  database,  a  comprehensive  personnel  record  system. 
For  this  study,  three  sets  of  two-digi„  numbers  were  randomly 
selected  for  each  of  the  six  officer  ranks.  The  sets  of  numbers  were 
then  matched  to  the  last  two  digits  of  the  social  security  numbers  of 
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all  company  and  field  grade  officers  with  Air  Force  Specialty  Codes 
(military  occupations)  other  than  87XX  through  99XX.  These  codes 
were  excluded  from  the  sample  because  officers  in  this  group,  such 
as  lawyers  and  physicians,  have  a  different  evaluation  system. 

From  the  resulting  list  of  officers,  1,526  officers  were  randomly 
selected  A  sample  size  of  1,526  was  based,  in  part,  on  the  desired 
confidence/  reliability  of  the  survey  results.  "A  confidence/reliability 
level  of  95%  ±  5%  for  survey  results  is  the  minimum  one  normally 
specified  and  desired  by  all  professional  surveying  organizations'' 
(Department  of  the  Air  Force,  1974,  p.  13).  Using  the  formula 
supplied  by  Air  Force  resources,  it  was  determined  that  a  sample 
size  of  763  surveys  would  be  needed  to  meet  the  95%  level. 
Anticipating  a  response  rate  of  50%,  1526  surveys  were  mailed.  As 
Table  1  shows,  sampling  was  stratified  on  the  basis  of  percentages  of 
officers  in  each  rank  in  the  population. 

Measures 

A  questionnaire  was  mailed  to  each  of  1,526  Air  Force  line 
officers  assigned  worldwide.  A  mailed  survey  was  the  most  efficient 
method  of  gathering  the  data  considering  the  size  and  geographic 
distribution  of  the  population. 

The  49-item  questionnaire  contained  9  demographic  items, 
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TABLE  1 

Stratified  Random  Sample  by  Rank 


Rank 

SnnvysMmW 

% 

Population 

2Lt 

10.43 

124 

1 Lt 

14.11 

168 

Capt 

39.89 

474 

Maj 

18.62 

398 

LtCol 

11.70 

250 

Col 

5.25 

112 

100.00 

1,526 
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9  items  collecting  self-reports  of  past  and  current  appraisals,  and 
35  opinion  items.  The  complete  questionnaire  may  be  found  in 
Appendix  H. 


Demographic  items 

Rank.  The  item  asked,  "What  is  your  current  rank?"  Response 
choices  were  1)  second  lieutenant,  2)  first  lieutenant,  3)  captain, 

4)  major,  5)  lieutenant  colonel,  6)  colonel,  and  7)  other. 

Gender.  The  officer  was  asked,  "What  is  your  gender?" 

Response  choices  were  1)  female,  and  2)  male. 

Major  command  assigned.  The  respondent  was  asked,  "To  what 
major  command  or  headquarters  are  you  currently  assigned?" 
Response  choices  were  1)  Air  Force  Logistics  Command  (AFLC), 

2)  Space  Command  (SPACECMD),  3)  Air  Force  Systems  Command 
(AFSC),  4)  Air  Training  Command  (ATC),  5)  Military  Airlift 
Command  (MAC),  6)  Strategic  Air  Command  (SAC),  7)  Pacific  Air 
Force  (PACAF),  Tactical  Air  Command  (TAC),  and  U.  S.  Air  Forces 
in  Europe  (USAFE),  8)  Headquarters  U.S.  Air  Force  (HQUSAF),  and 
9)  Other. 

Organizational  level.  The  item  asked,  "To  which  organizational 
level  are  you  assigned?"  Response  choices  were  1)  below  MAJCOM 
level,  2)  at  MAJCOM  level,  and  3)  above  MAJCOM  level.  Those 
assigned  to  Air  Force  detachments,  wings  and  other  military 
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organizations  that  report  to  a  major  command  headquarters  would 
fall  in  the  "below  MAJCOM  level"  category.  Officers  assigned  to 
Headquarters  U.S.  Air  Force  and  some  governmental  agencies 
would  fall  in  che  "above  MaJCOM  level"  category. 

Aeronautical  rating.  The  officer  was  asked,  "What  is  your 
aeronautical  rating?"  Response  choices  were  1)  no  aeronautical 
rating,  2)  navigator,  3)  pilot,  and  4)  other  aeronautical  rating. 

Race.  The  respondent  was  asked,  "What  is  your  racial  or  ethnic 
background?"  Response  choices  were  1)  American  Indian, 

2)  Black/Black  American/  Afro-American,  3)  Caucasian/White  (other 
than  Spanish  speaking),  4)  Oriental/Oriental  American  (Asian 
American,  Chinese,  Filipino,  Japanese,  Korean),  5)  Spanish 
speaking  origin  (Chicano,  Cuban,  Latin  American,  Mexican),  and 
6)  other. 

Commissioned  service.  The  item  asked,  "How  many  years  of 
commissioned  service  have  you  completed?"  Response  choices  were 
1)  Less  than  1  year,  2)  1-5  years,  3)  6-10  years,  4)  11-15  years, 

5)  16-20  years,  and  6)  21  or  more  years. 

Total  active  federal  military  service.  The  officer  was  asked, 

"How  many  years  of  total  active  federal  military  service  (TAFMS) 
have  you  completed?"  Responses  to  this  item  were  identical  to  the 
commissioned-service  item.  A  difference  between  the  two  items 
indicates  the  officer  served  on  active  duty  prior  to  commissioning. 
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Front-side  ratings.  This  item  asked  the  respondent  how  he  or 
she  was  rated  on  the  ten  performance  factors  on  the  front  side  of  the 
last  Officer  Effectiveness  Report  received  under  the  OER  system.  The 
officer  was  asked,  "Did  you  receive  well  above  standard  (i.e.,  "fire- 
walled")  ratings  for  all  ten  front-side  performance  factors?" 

Response  choices  were  1)  Yes,  2)  No,  and  3)  I  was  never  evaluated 
under  the  previous  OER  system.  This  item  was  used  to  obtain  a  self- 
report  measure  of  the  respondent's  last  OER  rating  under  the 
previous  OER  system. 

Back-side  ratings.  This  item  asked  how  the  officer  was  rated  on 
the  "evaluation  of  potential"  on  the  back  side  of  the  last  Officer 
Effectiveness  Report  received  under  the  previous  OER  system.  The 
officer  was  asked,  "Did  you  receive  all  top  block  ratings  (from  rater, 
additional  rater,  and  endorser)  on  the  back-side  Evaluation  of 
Potential  rating  scale?"  Response  choices  were  1)  Yes,  2)  No,  and 
3)  I  was  not  evaluated  under  the  previous  OER  system.  This  item 
was  also  used  to  obtain  a  self-report  measure  of  the  respondent  s  last 
OER  rating. 

Endorsement  level.  The  officer  was  asked,  "What  endorsement 
level  did  you  receive?  (use  equivalent  civilian  rank  if  applicable)." 
Response  choices  were  1)  colonel  or  below,  2)  brigidier  general, 

3)  major  general,  4)  lieutenant  general,  5)  general  or  higher,  and 
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6)  not  evaluated  under  the  OER  system.  Under  the  previous  OER 
system,  OER  endorsement  levels  (i.e.,  rank  of  senior  officer 
endorsing  an  officer's  OER)  played  a  central  role  in  determining  an 
officer's  suitability  for  promotion  in  rank. 

Promotion  recommendation  received.  The  respondent  was 
asked,  "What  did  vou  receive  for  your  most  recent  promotion 
recommendation?"  Response  choices  were  1)  Definitely  Promote, 

2)  Promote,  3)  Do  Not  Promote  This  Board,  and  4)  I  have  not  been 
evaluated  for  promotion  under  the  new  system.  Officers  receiving  a 
"Definitely  Promote"  recommendation  are  virtually  assured 
promotion  to  the  next  rank.  Captains,  for  example,  receiving  a 
"Promote"  recommendation  have  about  a  50%  chance  of  being 
promoted.  This  item  was  used  to  differentiate  officers  by  the  OES 
promotion  recommendation  they  received. 

Promotion  recommendation  expected.  The  item  asked,  "What  do 
vou  expect  to  receive  for  your  next  promotion  recommendation?” 
Response  choices  were  1)  Definitely  Promote,  2)  Promote,  3)  Do  Not 
Promote  This  Board,  and  4)  I  will  not  be  evaluated  under  the  new 
system  (retiring,  resigning,  etc.).  This  item  was  used  to  differentiate 
officers  by  the  promotion  recommendation  they  expected  to  receive 
under  the  OES. 
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Opinions  About  PER  and  OES 


The  survey  contained  35  items  measuring  opinions  about  aspects 
of  both  the  previous  OER  system  and  the  current  OES.  Each  item  was 
rated  on  a  7-point  rating  scale  ranging  from  (1)  strongly  disagree  to 
(7)  strongly  agree. 

The  Erst  ten  opinion  items  to  be  examined  compared  the  opinions 
of  officers  responding  to  Pontiffs  (1987)  study  on  the  OER  system  to 
identical  items  referencing  the  OES  in  the  current  study.  The  ten 
items  were  extracted  from  Pontiffs  (1987)  study.  They  were  taken 
verbatim,  with  the  exception  of  the  necessary  change  in  stipulating 
the  evaluative  referent  (i.e.,  OES  replaced  OER). 

Selecting  best  qualified.  Pontiff  (1987)  asked  his  srmple  of 
respondents  to  evaluate  the  statement  "OER  forms  and  other 
documents  in  promotion  folders  permit  promotion  boards  members 
to  select  the  best  qualified."  Officers  were  being  asked  to  evaluate  the 
utility  of  documents  used  by  promotion  boards  to  select  officers  for 
promotion  (e.g.,  OER,  personal  photo,  etc.).  The  current  survey 
contained  a  similar  item  referencing  OES  documents. 

Identify  performance.  Pontiff  (1987)  used  a  senes  of  five  items  to 
measure  the  capacity  of  the  OER  to  reflect  various  levels  of 
performance.  Similar  items  were  incorporated  in  the  current  survey 
to  gauge  the  ability  of  the  OES  to  reflect  varying  levels  of  officer  duty 
performance.  These  items,  as  they  originally  appeared  in  the  Pontiff 
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survey,  are  as  follows: 

1)  The  item  Well-Below  Average  Performance  Identified"  asked 
the  officer's  opinion  about  the  statement,  "An  officer  whose  duty 
performance  is  well  below  average  is  easily  identified  under  the 
current  OER  system.” 

2)  The  item  "Below  Average  Performance  Identified"  asked  the 
respondent’s  opinion  about  the  statement,  "An  officer  whose  duty 
performance  is  below  average  is  easily  identified  under  the  current 
OER  system". 

3)  The  item  "Average  Performance  Identified"  sought  an 
opinion  about  the  statement,  "An  officer  whose  duty  performance  is 
average  is  easily  identified  under  the  current  OER  system.” 

4)  The  item  "Above  Average  Performance  Identified"  asked  the 
officer's  opinion  about  the  statement,  "An  offic  whose  duty 
performance  is  above  average  is  easily  identified  under  the  current 
OER  system." 

5)  The  item  "Well-Above  Average  Performance  Identified"  asked 
the  respondent's  opinion  about  the  statement,  "An  officer  whose  duty 
performance  is  well  above  average  is  easily  identified  under  the 
current  OER  system." 

Pontiff  found  that  the  majority  of  officers  agreed  the  OER  system 
was  capable  of  identifying  those  officers  whose  duty  performance  was 
at  the  two  extremes  (well  below  average  and  well  above  average),  but 
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the  OERs  capability  degraded  when  attempting  to  identify  duty 
performance  that  lay  in  between  these  two  extremes. 

Time  spent  on  OPR.  Pontiff  (1987)  employed  three  items  to 
measure  concerns  about  the  amount  of  time  spent  completing  Officer 
Effectiveness  Reports.  Similar  items  evaluating  time  spent  on  the 
OES's  Officer  Performance  Report  (OPR)  were  incorporated  in  the 
current  study.  Pontiff  (1987)  concluded  that  officers  felt  "the  time  is 
well  spent;  however,  they  (the  officers)  would  dearly  like  to  see  a 
reduction  in  the  noncritical  administrivia’"  (  p.  15).  The  three 
items,  as  they  appeared  in  the  current  study,  are  shown  below. 

1)  The  item  "OPR  Appearance"  asked  the  officer  to  give  his  or 
her  opinion  on  the  statement,  "More  time  is  spent  on  OPR 
appearance  than  content." 

2)  The  item  "Time  on  OPR  Process"  sought  an  opinion  on  the 
statement,  "Too  much  time  is  taken  up  by  the  entire  OPR  process 
(from  initial  draft  to  becoming  a  macter  of  record).” 

3)  The  item  "Value  of  Time  on  OPRs"  asked  the  respondent  to 
evaluate  the  statement, "  The  time  needed  to  prepare  OPRs  is  weli 
worth  the  effort." 

Personal  input  into  OPR.  Pontiff  (1987)  asked  the  officer's 
opinion  of  the  statement,  "I  should  have  more  input  into  my  OER 
before  it  becomes  a  matter  of  record."  Pontiff  found  mixed  responses 
to  this  item,  with  almost  half  (47%)  of  the  officers  in  his  study 
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choosing  the  neither  agree  nor  disagree  response.  A  similar  item 
referring  to  the  OPR  was  embedded  in  the  current  survey. 

Pairs  of  items  were  used  in  the  current  survey  to  gather 
comparable  response  evaluations  of  the  OER  and  OES. 

PER  and  OES  promotion  fairness.  Officers  in  the  current  study 
were  asked  to  evaluate  the  fairness  of  promotion  decisions  under  the 
two  evaluation  systems. 

1)  The  item  "OER  Promotion  Decisions’'  asked  the  officer  s 
opinion  about  the  statement,  "Promotion  decisions  were  fair  and  just 
under  the  previous  OER  system." 

2)  The  item  "OES  Promotion  Derisions"  asked  the  respondent  s 
opinion  about  the  statement,  "Promotion  derisions  are  fair  and  just 
under  the  current  OES." 

PER  and  OES  appraisal  fairness.  Respondents  to  the  current 
survey  were  asked  their  opinion  about  the  fairness  of  performance 
appraisals  under  the  two  systems. 

1)  The  item  "OER  Appraisal  Fairness"  asked  the  officer  to 
evaluate  the  statement,  "Performance  was  fairly  and  accurately 
appraised  under  the  previous  OER  system." 

2)  The  item  "OES  Appraisal  Fairness"  asked  the  respondent  to 
evaluate  the  statement,  "Performance  is  fairly  and  accurately 
appraised  under  the  current  OES." 

OER  and  OES  feedback  clarity.  In  the  current  study,  officers 
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were  asked  their  opinion  about  the  ciarity  and  timeliness  of  feedback 
under  the  two  systems. 

1)  The  item  "OER  Clarity"  asked  the  officer's  opinion  about  the 
statement,  "Under  the  previous  OER  system  officers  received  clear 
and  timely  feedback." 

2)  The  item  "OES  Clarity"  asked  the  respondent  s  opinion  about 
the  statement,  "Under  the  current  OES  officers  receive  clear  and 
timely  feedback." 

Quality  of  feedback.  There  were  ten  items  that  examined 
feedback  under  the  current  system.  The  purpose  was  to  determine  if 
officers  were  receiving  formal  feedback  in  accordance  with  the 
guidelines  established  by  Air  Force  Pamphlet  36-6,  USAF  Officer's 
Guide  to  the  Officer  Evaluation  System.  Two  of  the  ten  items 
evaluated  perceptions  of  payoffs  for  feedback  relative  to  the  officer's 
job.  The  respondents  were  instructed  to  give  their  opinion  on  the  ten 
items  only  if  they  had  received  feedback  from  a  rater  using  the 
Performance  Feedback  Worksheet.  About  one  half  (i.e.,  444  of  906 
respondents)  of  the  sample  had  received  feedback. 

1)  The  item  "Specific  Feedback"  asked  the  respondent's  opinion 
about  the  statement,  "Specific  examples  of  my  performance  were 
cited." 

2)  The  item  "Objective  Feedback"  asked  the  officer  to  evaluate  the 
statement,  "The  comments  concerning  my  performance  were 
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objective." 

3)  The  item  "Two-Way  Communication"  sought  an  opinion  on 
the  statement,  "  There  was  two-way  communication  between  my 
rater  and  me." 

4)  The  item  "Balanced  Feedback"  asked  the  officer  to  evaluate  the 
statement,  "Both  my  strengths  and  weaknesses  were  discussed." 

5)  The  item  "Rater  Responsiveness"  sought  an  opinion  about  the 
statement,  "My  rater  listened  to  what  I  had  to  say  in  response  to  his 
feedback." 

6)  The  item  "Performance  and  Officership"  asked  the 
respondent's  opinion  about  the  statement,  "The  focus  was  on  duty 
performance  and  officership." 

7)  The  item  "Scope  of  Feedback"  asked  the  officer  s  opinion  about 
the  statement,  "The  examples  of  my  behavior  cited  by  the  rater  were 
drawn  from  the  entire  rating  period,  not  just  recent  events." 

8)  The  item  "Constructive  Feedback"  sought  an  opinion  about  the 
statement,  "My  rater  and  I  discussed  a  course  of  action  for  improved 
performance  for  my  next  feedback  session." 

9)  The  item  "Understand  Job"  asked  the  respondent  s  opinion 
about  the  statement,  "As  a  result  of  the  latest  feedback  session,  I  have 
a  better  understanding  of  what  is  expected  of  me." 

10)  The  item  "Ability  to  do  Job"  asked  the  officer  s  opinion  about 
the  statement,  "As  a  result  of  the  latest  feedback  session,  I  will  be 
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able  to  do  my  job  better." 

Quality  of  PFW.  There  were  three  items  dealing  with  the  ease  of 
completion  and  value  of  the  Performance  Feedback  Worksheet 
(PFW).  Officers  were  told  to  respond  to  the  three  items  only  if  they 
had  provided  feedback  as  a  rater  using  the  worksheet. 

1)  The  item  "Ease  of  PFW  Completion"  asked  the  officer’s 
opinion  about  the  statement,  "The  format  of  the  PFW  makes  it  easy  to 
complete." 

2)  The  item  "Useful  Rating  Scales"  sought  an  opinion  about  the 
statement,  "The  rating  scales  are  useful  in  portraying  how  much 
improvement  is  needed  in  an  officer  s  performance." 

3)  The  item  "Value  of  PFW'  asked  the  respondent’s  opinion 
about  the  statement,  "The  PFW  is  a  valuable  tool  for  conducting  a 
well  organized  feedback  session." 

Ease  of  OPR  completion.  Officers  were  asked  their  opinion  about 
the  statement,  "The  OPR  is  easier  to  complete  than  its  predecessor, 
the  OER."  Officers  were  to  respond  to  this  and  the  next  item  only  if 
they  had  completed  an  Officer  Performance  Report  as  a  ratfii. 

Value  of  OPR.  This  item  asked  the  respondent’s  opinion  about 
the  statement,  "The  OPR  is  a  better  tool  for  documenting  job 
performance  than  the  OER." 

Feedback  for  Lts  and  Capts.  Formal  feedback  is  mandatory  for 
lieutenants  and  captains.  To  judge  the  perceived  accuracy  of 
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feedback  for  this  group  of  officers,  respondents  were  asked  their 
opinion  about  the  statement,  "The  Officer  Evaluation  System  (OES) 
provides  lieutenants  and  captains  with  accurate  feedback  on  their 
duty  performance." 

Accuracy  of  assessment.  To  determine  if  officers  feel  their  duty 
performance  is  being  accurately  appraised  under  the  OES,  they  were 
asked  their  opinion  about  the  statement,  "The  OES  provides  me  with 
an  accurate  assessment  of  my  duty  performance  and  potential  based 
upon  that  performance.” 

Senior  rater  qualification.  As  a  result  of  the  new  decentralized 
promotion  process,  the  senior  rater  has  a  more  direct  role  in 
determining  a  subordinate's  chance  of  being  promoted.  This  item 
was  designed  to  find  out  how  much  officers  trust  their  senior  rater 
with  the  statement,  "My  senior  rater  is  better  qualified  to  determine 
my  promotion  potential  than  a  central  promotion  board." 

Organization  affecting  promotion.  Because  "Definitely  Promote" 
recommendations  are  allocated  according  to  manning  levels,  some 
small  units  do  not  receive  any.  Officers  from  these  small  units  meet 
intermediate  promotion  boards  at  the  next  higher  organization  level. 
To  determine  if  officers  feel  this  could  affect  their  chances  of 
receiving  a  "Definitely  Promote”  recommendation,  respondents  were 
asked  their  opinion  about  the  statement,  "An  officer  has  an  equal 
chance  of  receiving  a  "Definitely  Promote"  recommendation 
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regardless  of  the  organizational  level  to  which  he  or  she  is  assigned." 


Factor  Analysis 

For  the  purposes  of  data  reduction,  factor  analysis  was 
performed  on  the  35  opinion  items.  The  resulting  scales  were  then 
subjected  to  reliability  analysis  using  Cronbach’s  alpha.  The  results 
of  this  analysis  are  shown  in  Table  2. 

Data  reduction  was  especially  useful  when  examining 
differences  of  opinion  between  population  subgroups  for  sill  35  opinion 
items.  For  other  analyses,  data  collected  on  the  individual  items  were 
more  appropriate. 


Bittcedure 

Survey  questionnaires  were  mailed  to  1,526  Air  Force  line 
officers  worldwide.  Instructions  on  the  form  assured  the  respondent 
that  all  information  he  or  she  provided  would  be  held  in  strictest 
confidence.  Keywords  used  throughout  the  questionnaire  were 
defined  to  avoid  confusion.  Respondents  were  asked  to  return  the 
completed  optical-scanning  form  within  one  week  of  receipt. 

Comparison  of  the  Current  Sample  with  Pontiffs  (1987)  Sample 

Of  the  1,526  surveys  mailed  in  the  current  study,  906  were 
returned  for  a  response  rate  of  59.3  %.  Of  the  906  respondents,  59.8% 
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TABLE  2 

Factor  Analysis  Results  and  Reliability  Coefficients 


Item 

Factor  Name 

Cronbach's 

alpha 

Feedback  for  Lts  &  Capts 

Assessment  of  Performance 
Selecting  Best  Qualified 

Well-Below  Avg.  Perf.  Ident. 

Below  Average  Perf.  Ident. 

Average  Performance  Ident. 

Above  Average  Perf.  Ident. 
Well-Above  Avg.  Perf.  Ident. 

Identify  8c  Select  Best 

.87 

OPR  Appearance 

Time  on  OPR  Process 

Value  of  Time  om  OPRs 

Time  Spent  on  Forms 

.72 

OER  Promotion  Decisions 

OER  Appraisal  Fairness 

OER  Clarity 

Perception  of  OER 

.70 

OES  Promotion  Decisions 

OES  Appraisal  ’•'aimess 

OER  Clarity 

Perception  of  OES 

.76 

Specific  Feedback 

Objective  Feedback 

Two-Way  Communication 

Balanced  Feedback 

Rater  Responsiveness 

Performance  and  Officership 

Scope  of  Feedback 

Constructive  Feedback 
Understanding  Job 

Anility  to  do  Job 

Quality  of  Feedback 

.94 

Ease  of  PFW  Completion 

Useful  Rating  Scales 

Value  of  PFW 

Quality  of  PFW 

.84 
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were  company  grade  officers  and  40. 2%  were  field  grade  officers.  In 
Pontiffs  (1987)  study,  1,518  surveys  were  mailed  and  981  responded 
for  a  response  rate  of  64.6%.  Of  the  981  officers  participating  in 
Pontiff  s  study,  47%  were  company  grade  officers  and  53%  were  field 
grade  officers.  Currently,  64.4%  of  all  officers  in  the  USAF  are 
company  grade  officers  and  35.6%  are  field  grade  officers. 

Chi-square  tests  (for  two  independent  samples)  compared 
Pontiff  s  sample  with  the  current  study's  sample  on  items 
measuring  Grade  (company  grade  or  field  grade),  Rank,  Gender, 
Race,  and  Aeronautical  Rating.  The  purpose  of  these  tests  was  to 
determine  if  the  two  samples  were  significantly  different  in 
composition.  In  addition,  to  see  if  the  current  study's  sample  was 
significantly  different  from  the  overall  Air  Force  population,  similar 
tests  were  performed  on  the  same  measures. 

Comparison  of  PER  and  OES 

Two  analyses  compared  perceptions  of  the  previous  Officer 
Effectiveness  Reporting  (OER)  system  and  the  Officer  Evaluation 
System  (OES).  The  first  analysis  involved  comparison  of  the 
responses  to  items  in  Pontiff  s  (1987)  study  referencing  the  OER  with 
responses  to  similar  items  in  the  current  study  referencing  the  OES. 
A  second  analysis  evaluated  responses  of  the  current  sample  to  pairs 
of  items  asking  the  officer’s  opinion  about  aspects  of  both  systems. 
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To  analyze  officer  perceptions  about  various  features  of  the  OES, 


the  distribution  of  responses  across  the  7-point  rating  scale  for 
certain  items  was  tabulated.  The  goal  was  to  determine  the  majority 
opinion  of  officers  in  the  sample. 

Comparison  of  Subgroups 

Differences  in  opinion  between  subgroups  may  identify 
unanticip  ated  biases  with  respect  to  the  OES.  Statistical  tests  were 
performed  on  key  subgroups  of  the  sample  to  identify  significant 
differences.  For  data  reduction  purposes,  scales  based  on  factor 
analysis  results  were  used  in  these  analyses. 


48 


III.  Results 


Comparison  of  Pontiff  s  (1987)  Sample  with  the  Current  Sample 

Current  study  responses  to  the  demographic  items  measuring 
Grade  (i.e.,  company  grade  or  field  grade),  Rank,  Gender,  Ethnic 
Origin,  and  Aeronautical  Rating  were  compared  with  Pontiff  s  (1987) 
study.  In  addition,  comparisons  were  made  between  the  present 
sample  and  USAF-wide  manpower  statistics  on  these  same 
measures.  Chi-square  tests  (two  independent  samples)  were 
performed  to  determine  whether  the  composition  of  the  present 
sample  was  different  from  Pontiff s  sample  or  the  Air  Force 
population  as  a  whole.  Table  3  provides  the  results  of  this  analysis. 
Given  the  sample  sizes  and  the  high  degree  of  statistical  power  in  the 
study,  a  conservative  level  of  statistical  significance  (i.e.,  p  <  .01)  was 
used  throughout  the  study. 

Tests  comparing  Pontiff  s  study  and  the  current  study  produced 
significant  differences  on  measures  of  Grade,  Rank,  and  Ethnic 
Origin.  A  higher  percentage  of  field  grade  officers  responded  to 
Pontiffs  study  than  to  the  current  study.  Also,  a  lower  percentage  of 
minority  group  officers  responded  to  Pontiff s  study.  Given  the 
evidence  of  differences  between  the  current  study  and  Pontiff  s  (1987) 
sample,  comparisons  between  the  two  sets  of  results  must  be 
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TABLE  3 

Chi-square  Tests  Comparing  Sample-Composition  Statistics 


Current 

PcnOff 

USAF 

Sanek 

SrnnfJe 

Variable 

% 

% 

A2 

% 

& 

Grads 

Company  grade 

59.8 

47.0 

31.10* 

64.4 

3.90 

Field  grade 

40.2 

53.0 

35.6 

Hank 

2Lt 

7.2 

6.0 

10.4 

lLt 

13.6 

11.0 

14.1 

Captain 

39.1 

30.0 

35.66* 

39.9 

8.14 

Major 

21.7 

25.0 

18.6 

LtCol 

12.8 

20.0 

11.7 

Colonel 

5.6 

8.0 

5.3 

Gender 

Female 

11.4 

9.0 

3.08 

11.3 

0.01 

Male 

88.6 

91.0 

88.7 

Ethnis  Origin 

Black 

5.1 

4.0 

5.5 

Caucasian 

88.7 

93.0 

12.50* 

92.2 

16.66* 

Oriental 

2.0 

1.0 

.... 

Other 

4.2 

2.0 

2.3a 

Aeronautical  Hating 
Non-Rated 

60.5 

55.0 

61.5 

Navigator 

11.6 

13.0 

7.84 

11. 5b 

0.19 

Pilot 

26.2 

31.0 

27.0b 

Other 

1.7 

1.0 

a  Includes  officers  with  oriental  ethnic  origin. 

b  Includes  officers  with  other  than  pilot  or  navigator  aeronautical  rating, 
•pc. 01. 
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tempered  by  the  recognition  that  the  samples,  while  randomly  drawn 
from  the  same  population,  were  nevertheless  not  entirely  identical. 

There  were  no  significant  differences  between  the  current  study 
and  statistics  for  the  Air  Force  population  on  measures  of  Grade, 
Rank,  Gender,  and  Aeronautical  Rating.  These  data  support  the 
conclusion  that  respondents  in  the  current  study  were  reasonably 
representative  of  the  Air  Force  as  a  whole.  The  current  sample  was 
significantly  different  from  the  Air  Force  population  on  the  basis  of 
ethnic  origin.  Hence,  conclusions  must  be  tempered  by  this 
knowledge. 


Results  of  Self-Reported  Appraisals 

Front-Side  Ratings 

Of  the  901  officers  responding  to  this  item,  92.5%  reported 
receiving  "dl  "fire-walled"  OER  ratings,  1.1%  had  not,  and  6.4% 
reported  never  to  have  been  evaluated  under  the  previous  OER 
system. 

Back-Side  Ratings 

Of  the  901  officers  responding,  92.7%  reported  all  "top-  block" 
OER  ratings,  0.9%  had  received  lower  ratings,  and  6.4%  had  never 
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been  evaluated  under  the  OER  system. 

The  resulting  high  percentage  of  officers  receiving  "firewalled" 
front-side  ratings  and  "top-block"  back-side  ratings  from  the  previous 
OER  system  supports  wide-spread  belief  that  leniency  error  had 
severely  eroded  the  ability  of  these  rating  scales  to  serve  as  useful 
standards  of  measurement  in  the  performance  appraisal  and 
evaluation  process. 

Endorsement  Level 

Of  the  902  officers  responding,  32.0%  reported  receiving  at  most  a 
colonel's  endorsement,  15.4%  received  a  brigidier  general's 
endorsement,  30.4%  a  lieutenant  general's  endorsement,  3.0% 
received  at  least  a  general  s  endorsement,  and  5.8%  reported  not  to 
having  been  evaluated  under  the  previous  OER  system. 

Promotion  Recommendation  Received 

Only  190  (21.0%)  of  the  906  officers  reported  to  have  been 
evaluated  for  promotion  under  the  new  OES.  Of  those  responding, 
55.2%  reported  receiving  a  "Definitely  Promote"  recommendation, 
37.4%  received  a  "Promote"  recommendation,  and  7.4%  received  a 
"Do  Not  Promote  This  Board"  recommendation.  The  small 
percentage  of  officers  responding  to  this  item  was  due  to  the  relatively 
low  number  of  promotion  boards  that  have  met  under  the  new 
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system.  To  date,  there  has  been  only  one  promotion  board  advancing 
company  grade  officers  to  field  grade  billets  under  the  OES. 


Promotion  Recommendation  Expected 

This  item  asked  officers  to  predict  the  recommendation  they 
expected  to  receive  prior  to  their  next  OES  promotion  board.  Of  the 
891  responding,  54.4%  believed  they  would  receive  a  "Definitely 
Promote”  recommendation,  34.2%  expected  to  receive  a  "Promote" 
recommendation,  3.3%  expected  to  receive  a  "Do  Not  Promote  This 
Board"  recommendation,  and  8.1%  reported  they  would  not  be 
evaluated  under  the  new  OES  (i.e.,  retiring,  resigning,  etc.). 

Analysis  of  Objectives 


Objective  1 

The  main  objective  of  the  study  was  to  determine  if  the  current 
OES  is  perceived  by  officers  as  an  improvement  over  the  previous 
OER  system  for  the  purposes  of  accurately  assessing  duty 
performance,  selecting  the  best  qualified  for  promotion,  and 
providing  feedback.  To  support  this  objective,  comparisons  were 
made  between  comparable  items  from  the  Pontiff  study  and  the 
current  study.  In  addition,  responses  from  the  current  study,  asking 
officers  to  evaluate  the  OES  and  the  OER  system,  were  compared. 
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Table  4  shows  the  results  of  responses  to  six  items  common  to  the 
two  studies.  Hypothesis  la,  which  predicted  that  officers  would 
perceive  the  current  OES  to  be  better  than  the  previous  OER  system  in 
identifying  levels  of  job  performance,  was  supported.  Officers 
appeared  to  favor  the  OES.  They  believed  that  all  officers,  from  those 
with  well-below  average  duty  performance  to  those  with  well-above 
average  duty  performance,  were  more  easily  differentiated  by  the 
new  system.  They  also  believed  th  e  OES  was  a  better  tool  for  assisting 

promotion  board  members  in  selecting  the  best  qualified. 

Table  5  shows  the  results  from  the  current  study  for  the  pairs  of 
items  used  to  compare  the  OER  and  OES.  Hypothesis  lb,  predicting 
differences  in  the  fairness  of  promotion  decisions  under  the  two 
systems,  was  supported.  Officers  rated  the  OES  significantly  higher 
than  the  OER  system  in  its  ability  to  foster  fair  promotion  decisions. 
Hypothesis  lc,  relating  to  the  fairness  and  accuracy  of  job 
performance  appraisals  under  the  OER  and  OES,  was  supported. 
Officers  favored  the  OES  in  its  ability  to  provide  fair  and  accurate 
appraisals.  Finally,  Hypothesis  Id,  predicting  differences  in  the 
quality  of  feedback  under  the  two  systems,  was  supported.  As 
evidenced  by  the  large  t-value,  there  was  a  substantial  difference  on 
perceived  feedback.  Officers  felt  they  received  more  clear  and  timely 


feedback  under  the  OES. 


TABLE  4 

T-Tests  Comparing  Ratings  on  Items  Common 
to  the  Current  Study  and  to  the  Pontiff  (1987)  Study 


Pontiff 

Current 

Sample 

Snmnle 

Item 

M 

SD 

M 

SD 

t 

Selecting  Best  Qualified 

3.76 

1.75 

450 

1.50 

6.82* 

Well-Below  Average  Performance  Ident. 

4.16 

1.92 

4.94 

1.53 

9.82* 

Below  Average  Performance  Identified 

3.46 

1.72 

4.55 

1.48 

14.81* 

Average  Performance  Identified 

2.97 

1.54 

4.18 

1.46 

17.60* 

Above  Average  Performa  ice  Identified 

3.40 

1.67 

4.38 

1.46 

13.50* 

Well-Above  Average  Performance  Ident. 

4.12 

1.87 

4.81 

1.59 

8.69* 

*p<  .001. 


55 


TABLE  5 

T-Tests  Comparing  OER  and  OES 


Ban 

M 

SD 

OER  Promotion  Fairness 

3.51 

1.54 

OES  Promotion  Fairness 

4.11 

1.30 

OER  Appraisal  Fairness 

3.14 

1.55 

OES  Appraisal  Fairness 

4.32 

1.35 

OER  Feedback  Clarity 

2.32 

1.22 

OES  Feedback  Clarity 

4.58 

1.45 

*  p  <  .001. 


t 


10.34  * 


19.45  * 


36.57  * 
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Aa  evidenced  by  the  data  from  both  sources,  the  OES  was 
perceived  by  Air  Force  line  officers  as  an  improvement  over  the 
former  OER  system  in  terms  of  its  ability  to  accurately  assess  duty 
performance,  select  the  best  qualified  for  promotion,  and  provide 
feedback. 

ObififitiYfi,  2 

Objective  2  was  to  determine  if  the  feedback  officers  were 
receiving  under  the  OES  was  in  accordance  with  Air  Force  Pamphlet 
36-6,  USAP  Officer's  Guide  to  the  Officer  Evaluation  System. 
Hypothesis  2,  predicting  that  a  majority  of  officers  agree  that  raters 
are  adhering  to  the  guidelines  on  feedback,  was  supported. 

Across  the  ten  items  in  Table  6  aimed  at  determining  the  quality 
of  feedback  being  received  during  formal  feedback  sessions,  a 
majority  (i.e.,  58.1%  to  82.2%)  of  officers  agreed  that  they  were 
receiving  feedback  in  accordance  with  the  guidelines.  The  content  of 
these  items  suggest  that  specific  examples  of  performance  were 
being  cited  from  the  entire  rating  period,  the  comments  were 
objective,  there  was  two-way  communication,  both  strengths  and 
weaknesses  were  being  discussed,  the  raters  were  listening  to 
feedback  from  the  ratees,  the  focus  was  on  duty  performance  and 
officership,  and  courses  of  improvement  were  being  discussed.  As  a 
result  of  these  feedback  sessions,  the  ratees  felt  they  had  a  better 
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TABLE  6 

Distribution  of  Opinions  on  Cuirent  Feedback 


DisAtrree 

Neutral 

Agree 

Item 

% 

% 

% 

N 

Specific  Feedback 

15.5 

4.8 

79. 7a 

444 

Objective  Feedback 

11.3 

8.7 

80.0a 

444 

Two-Way  Communication 

13.1 

4.7 

82.2a 

444 

Balanced  Feedback 

15.5 

5.9 

78. 6P 

444 

Rater  Responsiveness 

12.6 

9.7 

77. 7a 

444 

Performance  &  Officership 

10.0 

8.1 

81. 9a 

442 

Scope  of  Feedback 

16.5 

9.3 

73. 6a 

443 

Constructive  Feedback 

27.3 

12.6 

60.  la 

439 

Understanding  Job 

20.6 

14.8 

64. 6a 

441 

Ability  tc  Do  Job 

22.2 

19.7 

58.  la 

437 

a  percentage  of  responses  exceeds  50%. 


understanding  of  their  jobs  and  believed  they  would  be  able  to  do  their 
jobs  better. 


Objective  3 

Increased  responsibility  for  senior  raters  in  the  promotion 
process  was  an  additional  change  instituted  with  the  introduction  of 
the  revised  OES.  To  examine  officer  attitudes  toward  this  change, 
officers  were  asked  who  was  better  qualified  to  determine  their 
promotion  potential.  Hypothesis  3,  which  predicted  that  a  majority  of 
officers  would  feel  their  senior  rater  was  more  qualified  than  a 
central  promotion  board,  was  supported.  A  majority  (72.4%)  of  the 
904  respondents  favored  their  senior  rater,  16.3%  favored  a  central 
promotion  board,  and  11.3%  had  no  preference. 

Objective  4 

Objective  4  sought  to  determine  if  officers  believed  they  have  an 
equal  chance  of  receiving  a  "Definitely  Promote'"  recommendation 
regardless  of  the  organizational  level  to  which  they  were  assigned. 
Hypothesis  4,  which  predicted  that  a  majority  of  officers  would  feel 
they  have  an  equal  chance  of  receiving  a  "Definitely  Promote" 
recommendation,  was  not  supported.  A  mujority  (64.8%)  of  the 
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fourth  (23.3%)  of  the  904  respondents  thought  they  had  an  equal 
chance,  and  11.9%  were  undecided. 

Obiective  5 

The  OES  revision  introduced  new  forms  for  completion.  The 
purpose  of  objective  5  was  to  determine  how  officers  felt  about  the 
utility  of  the  two  new  forms,  the  Performance  Feedback  Worksheet 
(PFW)  and  the  Officer  Performance  Report  (OPR).  The  results  of  this 
analysis  are  shown  in  Table  7. 

The  hypothesis  (i.e.,  Hypothesis  7a)  stating  that  the  PFW  is  easy 
to  complete  and  a  valuable  tool  was  supported.  Between  71.4%  and 
81.1%  of  approximately  300  officers  surveyed  agreed  that  the  PFW  is 
easy  to  complete,  that  the  rating  scales  were  useful,  and  that  it  was  a 
valuable  tool  for  conducting  a  well-organized  feedback  session. 

Hypothesis  5b,  which  predicted  that  the  OPR  was  easier  to 
complete  and  better  than  the  OER  for  documenting  performance, 
was  supported.  Of  the  320  respondents  86.3%  agreed  that  the  OPR 
was  easy  to  complete  and  almost  70%  thought  the  OPR  was  a  better 
tool  than  the  OER  for  documenting  job  performance. 

Obiective  6 

To  determine  how  officers  felt  about  the  amount  of  time  spent 
completing  OES  forms  relative  to  the  amount  of  time  spent 
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TABLE  7 

Distribution  of  Opinions  on  OES  Forms 


DisAgrve 

Neutral 

Aspee. 

Item 

% 

% 

% 

N 

Ease  of  PFW  Completion 

10.9 

9.0 

80.  la 

302 

Useful  PFW  Rating  Scales 

15.3 

13.3 

71. 4a 

301 

Value  of  PFW 

10.3 

8.6 

81. la 

301 

Ease  of  OPR  Completion 

7.8 

5.9 

86.3a 

320 

Value  of  OPR 

17.2 

13.1 

69.7a 

320 

a  percentage  of  responses  exceeds  50%. 
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completing  OER  forms,  a  comparison  was  made  between  responses 
from  Pontiff s  (1987)  study  and  the  current  study.  Three  common 
items  were  used  in  both  studies.  The  results  are  shown  in  Table  8. 
Hypothesis  6,  predicting  that  there  were  no  differences  in  officer 
perceptions  of  the  amount  of  time  spent  filling  out  both  forms,  was 
not  supported.  Officers  in  the  current  study  were  significantly  less 
concerned  by  the  amount  of  time  spent  on  the  Officer  Performance 

Report  process  than  officers  in  the  Pontiff  study.  Current 
respondents  also  felt  the  time  needed  to  prepare  OPRs  was  more 
worthwhile  than  did  Pontiff  s  respondents. 

Objective  7 

The  purpose  of  objective  7  was  to  determine  if  there  were 
significant  differences  in  perceptions  of  the  OES  between  minority 
and  majority  subgroups  of  the  population.  No  significant  differences 
were  found  between  white  officers  and  nonwhite  officers  or  between 
males  and  females.  Hypothesis  7a,  which  predicted  that  there  would 

be  is  no  overt  evidence  of  discrimination  in  the  OES,  was  supported. 

Table  9  shows  there  were  significant  differences  in  the  views  of 
company  and  field  grade  officers.  The  prediction  (Hypothesis  7b) 

that  company  and  field  grade  officers  would  feel  no  differently  about 

the  OES  was  not  supported.  Company  grade  officers  were  more 
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TABLE  8 

T -Tests  Comparing  OER  and  OPR  Ratings 
from  Pontiff  (1987)  and  the  Current  Study 


Ryitiff  Current 

Sample  Sample 


Item 

M 

SD 

M 

SD 

t 

OPR  Appearance 

4.80 

1.69 

4.42 

1.65 

4.97* 

Time  on  OPR  Process 

5.00 

1.75 

4.61 

1.68 

4.99* 

Value  o i  Time  on  OPRs 

4.17 

1.77 

4.69 

1.48 

6.89* 

*p  <  .001. 
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TABLE  9 

T-Tests  Comparing  Responses  from  Company  Grade  and  Field  Grade  Officers 


Company  Field 

Grade  Grade 


Measure 

M 

SD 

M 

SD 

t 

Identify  &  Select  Best® 

4.64 

1.10 

4.61 

1.04 

0.40 

Time  Spent  on  Forms® 

4.21 

1.22 

3.96 

1.35 

2.90* 

Perception  of  Previous  OER® 

2.95 

1.14 

3.05 

1.15 

1.23 

Perception  of  Current  OES® 

4.42 

1.10 

4.22 

1.15 

2.56 

Quality  of  Feedback® 

5.18 

1.20 

4.40 

1.27 

3.94** 

Quality  of  PFW® 

5.19 

1.10 

5.17 

1.14 

0.10 

Ease  of  OPR  Completion^ 

5.60 

1.37 

5.71 

1.28 

0.73 

Value  of  OPRb 

5.13 

1.50 

5.00 

1.60 

0.77 

Personal  Input  Into  OPRb 

4.54 

1.51 

4.03 

1.62 

4.84** 

Senior  Rater  Qualification*5 

5.35 

1.49 

4.95 

1.70 

3.73** 

Org.  Affecting  Promotion*3 

3.11 

1.69 

2.96 

1.80 

1.28 

®  Multi-item  scales  based  on  factor  analysis. 
b  Individual  items. 

*p  <  .005.  **p  <  .001. 
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positive  about  the  performance  feedback  sessions  and  had  more  trust 
in  their  senior  rater  s  role  in  promotion  decisions  than  field  graie 
officers.  Company  grade  officers  were  more  concerned  than  field 
grade  officers  about  the  amount  of  time  spent  completing  OES  forms 
and  the  amount  of  input  they  had  into  their  OPR  before  it  became  a 
matter  of  record. 

Hypo  thesis  7c,  stipulating  that  nonrated  and  rated  officers  would 
feel  no  differently  about  the  OES,  was  not  supported.  Table  10  shows 
there  were  significant  differences  in  the  opinions  of  nonrated 
(officers  with  no  aeronautical  rating)  and  rated  officers. 

Nonrated  officers  had  a  more  favorable  view  of  the  current  OES 
and  felt  more  strongly  about  its  ability  to  identify  and  select  the  best 
qualified  than  did  rated  officers.  Rated  officers  were  more  concerned 
than  nonrated  officers  with  the  amount  of  time  spent  completing  OES 


forms. 


TABLE  10 

T -Tests  Comparing  Responses  from  Nonrated  and  Rated  Officers 


Measures 

Non-Rated 

M  SD 

Rated 

M  SD 

t 

Identify  &  Select  Besta 

4.72 

1.05 

4.49 

1.10 

3.24** 

Time  Spent  on  Forms4 

3.98 

1.22 

4.31 

1.35 

3.82** 

Perception  of  Previous  OERa 

3.07 

1.14 

2.87 

1.13 

2.55 

Perception  of  Current  OESa 

4.43 

1.09 

4.20 

1.17 

2.97* 

Quality  of  Feedback4 

5.15 

1.19 

4.99 

1.30 

1.33 

Quality  of  PFWa 

5.31 

1.06 

4.97 

1.20 

2.56 

Ease  of  OPR  Completion*3 

5.57 

1.43 

5.82 

1.12 

1.70 

Value  of  OPR*3 

5.06 

1.60 

5.02 

1.52 

0.24 

Personal  Input  Into  OPR*3 

4.38 

1.61 

4.28 

1.53 

0.92 

Senior  Rater  Qualification*3 

5.12 

1.62 

5.29 

1.52 

1.56 

Org.  Affecting  Promotion*3 

3.10 

1.71 

2.98 

1.77 

1.00 

a  Multi-item  scales  based  on  factor  analysis, 
b  Individual  Items 
*p  <  .005.  **p  <  .001. 
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IV.  Discussion 


Accurate  performance  appraisals  and  the  selection  of  the  best 
qualified  for  promotion  are  at  the  forefront  of  Air  Force  officers' 
concerns  regarding  their  performance  appraisal  system.  The  Air 
Force's  "up  or  out"  policy  puts  considerable  weight  on  the  outcome  of 
periodic  appraisals.  In  fact,  "there  is  no  parallel  in  private  industry 
whereby  one  performance  appraisal  can,  in  effect,  dictate  a  decision 
to  lay  off  a  person  many  years  in  the  future"  (Syllogistics,  Inc.  &  The 
Hay  Group,  1988,  IV- 11). 

The  main  objective  of  this  study  was  to  determine  if  officers 
perceive  the  new  OES  to  be  better  than  the  previous  OER  system  in 
accurately  assessing  duty  performance,  selecting  the  best  qualified 
for  promotion,  and  providing  constructive  feedback.  Responses  from 
two  random  samples  drawn  from  the  USAF  officer  population 
indicate  that  officers  perceive  the  new  OES  to  be  an  improvement  over 
the  earlier  OER  system  in  assessing  duty  performance  regardless  of 
the  level  of  that  performance.  The  data  indicate  that  respondents 
believed  that  promotion  boards  now  have  better  information  from 
which  to  select  the  best  qualified.  The  data  from  the  current  study 
comparing  the  OES  and  the  OER  indicate  that  officers  perceived  that 
job  performance  was  appraised  more  fairly  and  accurately  and 
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promotion  decisions  were  seen  as  fairer  under  the  revised  system. 

These  results  are  not  surprising  considering  the  changes  that 
took  place  under  the  new  system.  The  first  change  was  a 
modification  of  the  appraisal  form,  the  Officer  Performance  Report 
(OPR).  Under  the  new  system,  comments  on  completion  of 
professional  military  education,  advanced  academic  education,  and 
family  activities  are  not  to  be  included  on  the  OPR.  While  not 
diminishing  the  importance  of  these  activities,  the  Air  Force  did 
remove  them  from  the  OPR  to  persuade  raters  to  concentrate  on  job 
performance  behaviors  that  contribute  to  the  mission  of  the 
organization.  The  focus  on  job  performance  removed  one  of  the 
inequities  officers  felt  existed  under  the  old  system.  It  was  perceived 
that  some  officers  were  being  advanced,  not  based  upon  their 
contribution  to  the  mission,  but  on  the  volume  of  nonmission-related 
activities  they  had  completed  during  the  rating  period.  This  created 
internal  conflict  for  officers  who  were  not  as  adept  at  "gaming"  the 
system  but  who  felt  their  job  performance  was  exemplary. 

The  second  change  involved  decentralizing  the  promotion 
process.  The  senior  rater  now  has  greater  influence  on  an  officer's 
chances  for  promotion.  In  the  Syllogistics  &  Hay  Group  (1988)  study, 
most  officers  felt  more  control  was  needed  and  that  the  wing  level 
was  the  most  likely  place  to  implement  greater  control.  Findings 
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from  the  current  study  tend  to  strengthen  this  argument.  The 
majority  of  officers  agreed  that  their  senior  rater  was  better  qualified 
to  determine  their  promotion  potential  than  a  central  promotion 
board.  There  was  a  feeling  among  junior  officers,  in  particular,  that 
individuals  on  the  central  promotion  board  didn’t  have  time  for  an  in- 
depth  examination  of  an  officer's  records  (Syllogistics,  Inc.  &  The 
Hay  Group,  1988).  Officers  may  feel  that  more  care  will  be  taken  in 
promotion  decisions  made  at  the  unit  level  where  the  senior  rater  has 
a  greater  stake  in  the  success  of  his  or  her  own  officers. 

One  of  the  major  reasons  the  Air  Force  went  to  a  forced 
distribution  method  was  because  of  leniency  error  in  ratings.  In  the 
current  study,  92.5%  of  the  respondents  reported  receiving 
"fire walled"  front-side  ratings  and  92.7%  reported  "top-block"  back¬ 
side  ratings  on  their  last  OER.  These  results  clearly  confirm  the 
wide-spread  belief  that  the  old  system  was  incapable  of  coping  with 
the  leniency-error  tendencies  of  Air  Force  raters.  Under  the  new 
system,  55.2%  reported  receiving  "Definitely  Promote” 
recommendations  on  their  latest  promotion  board,  37.4%  reported 
receiving  a  "Promote”  recommendation,  and  7.4%  reported  receiving 
a  "Do  Not  Promote  This  Board"  recommendation.  Clearly,  the  forced 
distribution  method  underlying  the  OES  has,  in  effect,  reduced  the 
rampant  leniency  error  of  the  former  system.  It  is  also  informative 
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to  consider  the  data  focusing  on  officer  expectations  for  their  next 
promotion  recommendation.  Of  the  891  officers  responding,  54.4% 
expected  to  receive  a  "Definitely  Promote"  recommendation,  34.2% 
expected  to  receive  a  "Promote"  recommendation,  and  3.3%  expected 
to  receive  a  "Do  not  Promote  This  Board"  recommendation.  The 
characteristics  of  these  expectations  corresponded  very  closely  to  the 
distribution  of  promotion  recommendations  actually  doled  out  under 
the  OES.  Furthermore,  these  results  indicate  that  the  OES  has 
apparently  had  a  significant  effect  on  officer  expectations.  The  OES's 
forced  distribution  approach  has  not  only  resulted  in  actual  controls 
on  promotions,  it  has  also  affected  the  way  officers  think  about  their 
own  promotion  potential. 

Data  from  the  current  study  comparing  the  OER  and  the  OES 
indicated  that  officers  perceived  feedback  to  be  more  clear  and  timely 
under  the  OES.  Officers  in  the  current  study  rated  aspects  of  the 
feedback  they  received  from  the  OES-mandated  feedback  sessions. 
The  majority  of  officers  provided  favorable  ratings  on  the  feedback 
techniques  being  used,  techniques  that  the  Air  Force  encouraged 
raters  to  adopt  through  written  pamphlets  and  videotape  exercises. 

In  all  likelihood,  raters  have  benefited  from  the  feedback  training 
they  were  given  in  tandem  with  implementation  of  the  OES. 

To  increase  accuracy  in  ratings,  the  focus  of  contemporary 
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research  has  shifted  from  emphasis  on  rater-error  training  to 
training  raters  not  only  to  rate  fairly  but  to  observe  accurately 
(Bemadin  &  Pence,  1981).  The  Air  Force's  approach  to 
implementing  the  OES  has  emulated  this  trend  by  using  videotape 
seminars  which  focus  on  observing  and  reporting  job-related 
behavior  accurately. 

Studies  have  shown  that  the  positive  effects  of  rater  training 
programs  decrease  over  time  (Bemadin,  1978;  Ivancevich,  1979). 
Potentially,  the  benefits  of  recent  formal  feedback  sessions  were  a 
result  of  rater  interest  in  adapting  to  the  new  system  and  the 
extensive  training  provided.  In  light  of  evidence  indicating 
decreased  positive  effects  of  rater  training  over  time  (Bemadin  & 
Pence,  1981),  it  may  be  necessary  to  continue  active  rater  training  to 
maintain  high  levels  of  constructive  feedback  under  the  auspices  of 
the  OES  feedback  program. 

On  the  issue  of  promotion  opportunity,  the  majority  of  officers  felt 
they  did  not  have  an  equal  chance  of  receiving  a  "Definitely  Promote" 
recommendation  regardless  of  the  organizational  level  to  which  they 
were  assigned.  It  is  unfortunate  that  no  data  were  available  to 
determine  whether  this  feeling  existed  prior  to  the  OES.  One  possible 
explanation  for  this  finding  may  reside  in  the  method  used  to 
determine  "Definitely  Promote"  allocations.  Some  very  small  units 


71 


receive  no  allocations  and  must  forward  officers  for  promotion 
recommendations  to  the  next  higher  organizational  level.  This 
process  could  produce  a  perceived  disadvantage  for  the  affected 
officer. 

Another  concern  is  the  perceived  inequality  between  groups  at 
different  levels  in  the  organization.  Officers  may  feel  they  have  a 
better  chance  of  receiving  a  "Definitely  Promote"  recommendation  at 
a  base-level  assignment  where  the  competition  may  not  be  as  intense. 
At  headquarters  level  many  junior  officers  have  already  proven 
themselves  at  base  level  before  being  advanced  to  the  headquarters. 
Since  officers  assigned  to  a  headquarters  may  perceive  they  are 
competing  with  a  more  elite  group,  they  may  also  believe  this 
diminishes  their  chance  of  receiving  a  "Definitely  Promote" 
recommendation. 

Differences  in  perceptions  between  minority  and  majority 
subgroups  of  the  population  could  signal  deficiencies  in  the  OES,  an 
evaluation  system  intended  to  be  as  free  from  bias  as  possible.  In 
several  studies  examining  the  interaction  between  the  sex  of  the  rater 
and  the  sex  of  the  ratee  on  performance  appraisals,  no  systematic 
evidence  of  gender  bias  was  found  (Mobley,  1982;  Peters  et  al.,  1984,; 
Wexley  &  Pulakos,  1983).  In  a  meta-analysis  of  the  effects  of  a  ratee's 
race  on  performance  ratings,  supervisors  tended  to  give  higher 
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ratings  to  same-race  subordinates,  but  as  the  percentage  of  blacks 
increased  in  the  work  environment,  race  effects  decreased  (Kraiger  & 
Ford,  1985).  With  respect  to  the  OES,  no  differences  in  perceptions 
were  found  between  male  and  female  officers  and  between  white  and 
nonwhite  officers. 

When  comparing  perceptions  by  grade,  company  grade  officers 
had  a  more  positive  view  of  OES  feedback  sessions  than  did  field 
grade  officers.  These  results  were  in  line  with  the  Syllogistics  &  Hay 
Group  (1988)  study  which  determined  that  junior  officers  (i.e., 
company  grade)  expressed  the  strongest  desire  to  receive  feedback  on 
their  performance  from  their  immediate  supervisors.  In  the  current 
study,  company  grade  officers  were  also  more  in  favor  of  a 
decentralized  promotion  system  than  their  field  grade  counterparts. 

There  are  two  plausible  explanations  for  the  differences  between 
field  grade  officers  and  company  grade  officers  in  terms  of  their 
preference  for  decentralization  of  promotion  decisions.  First,  many 
field  grade  officers  remember  the  unpopular  "controlled"  OER 
system  between  1974  and  1978  and,  thus,  may  be  reluctant  to  accept  a 
new  system  that  features  controls  on  promotion  recommendations. 
Most  company  grade  officers  are  too  junior  to  have  been  affected  by 
the  "controlled"  OER  system.  Secondly,  most  field  grade  officers 
have  an  investment  in  the  older  system.  That  is,  they  have  spent  the 
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last  ten  years  preparing  for  evaluations  tinder  the  old  system  (i.e., 
"filling  squares")  and  may  be  reluctant  to  learn  a  new  set  of  rules 
this  late  in  their  careers. 

In  the  current  study,  company  grade  officers  were  also  more 
concerned  about  the  amount  of  time  spent  completing  OES  forms 
than  field  grade  officers.  It  may  be  that  the  burden  of  writing  OPRs 
is  greater  for  junior  grade  officers  who  are  generally  less 
experienced  at  writing  and  may  have  less  administrative  support 
than  field  grade  officers. 

When  comparing  the  OER  system  and  the  OES,  nonrated  officers 
were  more  positive  than  rated  (i.e.,  flying)  officers  about  the  new 
OES.  They  were  also  more  favorable  about  the  OES's  ability  to  identify 
and  select  the  best  qualified.  Perhaps  rated  officers,  especially  in  the 
junior  grades,  believe  that  it  is  harder  for  them  to  be  recognized  for 
their  performance.  In  a  flying  squadron  there  are  dozens  of  junior 
grade  officers  all  doing  the  same  thing,  flying.  Not  only  are  they 
competing  with  officers  in  the  same  squadron,  but  they  are  also 
competing  with  officers  in  other  flying  squadrons  at  the  same  base, 
as  well.  Nonrated  officers  of  the  same  rank  are  often  given  jobs 
where  they  have  direct  responsibility  for  more  people  and  resources, 
and  thus,  are  perceived  by  rated  officers  to  have  a  better  chance  or 
being  recognized  when  it's  time  to  make  promotion  decisions. 
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With  respect  to  the  amount  of  time  completing  OES  forms,  rated 
officers  were  more  concerned  about  the  time  spent  than  nonrated 
officers.  Many  nonrated  officers  work  in  an  environment  that 
supports  administrative  tasks,  like  completing  performance  reports, 
while  their  rated  counterparts  spend  much  of  their  time  away  from  a 
desk  either  flying  from  the  support  base  or  from  a  temporary  duty 
location.  Flight  duty  is  not  conducive  to  administrative  work  and 
may  be  the  reason  rated  officers  are  more  concerned  about  the 
amount  of  time  needed  to  complete  OES  forms. 

Study  Limitations 

The  most  reliable  results  comparing  officer  perceptions  of  the 
previous  OER  system  and  the  current  OES  would  be  obtained  by  a 
longitudinal  study  in  which  the  responses  of  the  same  individuals 
were  hacked  over  time.  Unfortunately,  thi.  me  lod  of  study  was  not 
feasible  in  the  current  instance.  The  current  study  compared  an 
independent  random  sample  from  1987  evaluating  the  OER  to  a 
second  random  sample  evaluating  the  OES.  Obviously,  the 
legitimacy  of  my  comparisons  hinges  on  the  deg  of  similarity  of 
the  two  samples  relative  to  their  respective  populations.  The  current 
sample  was  significantly  differen*  from  the  Pontiff  (1987)  study 
sample  in  terms  of  Grade,  Rank  and  Ethnic  Origin.  Conclusions 


drawn  from  comparisons  between  these  two  studies  must  be 
tempered  by  the  knowledge  that  the  samples  were  not  entirely 
parallel.  The  current  sample  was  significantly  different  from  the 
Air  Force  population  in  terms  of  Ethnic  Origin.  This  difference  must 
be  considered  when  drawing  conclusions  about  the 
representativeness  of  the  results,  as  well. 

Data  on  the  self-reported  ratings  for  the  items  Front-Side 
Ratings,  Back-Side  Ratings,  Endorsement  Level,  and  Promotion 
Recommendation  Received,  relied  on  frank  disclosure  by  the 
respondents.  The  validity  of  such  self-reports  must  be  considered 
when  drawing  conclusions  (see  e.g.,  Levine,  1980). 

Another  limitation  of  this  research  is  the  timing  of  the  study. 

The  questionnaires  were  sent  out  10  months  after  the 
implementation  of  the  new  Officer  Evaluation  System.  During  that 
time,  no  promotion  boards  for  officers  advancing  to  the  mere 
competitive  ranks  of  major,  lieutenant  colonel,  and  colonel  took 
place.  Because  the  'real'’  competition  had  not  yet  taken  place,  the 
data  on  the  OElS's  ability  to  identify  and  select  the  best  qualified  is 
based  largely  on  perceptions  of  what  officers  'think''  the  OES  is 
capable  of,  not  on  their  perceptions  following  the  results  of  numerous 
promotion  cycles.  Hence,  the  promotional  material  (e.g.,  videotapes) 
provided  to  the  officers  on  the  OES  may  have  haa  a  greater  impact  on 
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the  results  than  actual  experience  with  the  OES  itself.  Timing  was 
also  a  factor  for  the  444  officers  who  received  formal  feedback  under 
the  OES.  For  the  majority  of  them,  they  have  not  had  a  chance  to 
compare  what  their  superiors  are  saying  about  them  to  resulting 
promotion  decisions. 

Recommendations 

The  data  from  this  study  indicated  that  as  of  this  juncture  in 
time,  officers  perceive  the  OES  as  an  improvement  over  the  older  OER 
system.  Further  research  is  warranted.  The  reasons  why  officers 
did  not  feel  they  had  an  equal  chance  of  receiving  a  "Definitely 
Promote"  recommendation  regardless  of  their  unit  of  assignment 
should  be  investigated.  It  is  important  not  only  to  determine  why 
they  feel  this  way  but  also  to  gather  the  promotion  data  to  confirm  or 
deny  their  fears.  Differences  in  perceptions  between  company  grade 
and  field  grade  officers  should  be  studied  to  determine  why  field 
grade  officers  are  less  enthusiastic  about  the  new  OES.  Differences 
in  perception  between  nonrated  and  rated  officers  should  also  be 
investigated  to  determine  whether  those  differences  are  due  to 
aspects  of  the  new  OES  or  intervening  variables  not  associated  with 
the  OES.  All  of  these  issues  warrant  further  study.  To  draw  more 
concrete  conclusions  about  perceptions  of  the  new  OES,  a 


longitudinal  study  should  be  conducted  downstream  to  allow  time  for 
the  OES  to  "settle  in"  and  become  institutionalized.  Particular 
attention  should  be  paid  to  the  level  of  rater  training  and  the  quality  of 
feedback  received  during  this  period  to  determine  whether  or  not  a 
long-term  rater  training  program  should  be  institutionalized,  as 
well. 
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Appendix  A:  U.S.  Armv  Officer  Evaluation  Report 


OMlCtft  CVACUATtON  REPORT  SUPPORT  FOAM 

P«r  «•  «rf  tft*  f«r<n.  A  A  tOA  tf**  »Miwm  OCI/M 


*»  wa  R#wm*  >#f»A  Com0*mt\*4  i*m  form 


nami  op  ratio  oppicir  amt.  7*mt  mi> 


PART  I  -  RATIO  OPPICIR  iQSMTlPlCATlQM 


OMOAHi|AT>ON 


PART  II  .  RAT1M0  CHAIM  -  TOUR  RAT  I  M3  CHAIM  POA  THf  IVAIUATIQM  PfRtQO  If 


RATIR 


l  MTS  RMS  01  ATI 
RATIR 


SI  Ml  OR 
RATIR 


!mam| 


QMAOI 


rosiT'ON 


PART  III  rn  VlRIPjCATtOM  OP  INITIAL  PAd  TQAACI  OtSCURSlQM 


AM  >MlTlAt  PACI-TO-PACI  Ot*CUM>OM  OP  OOTlft.  ««  MOMftd  i  L  i  Ti  1 1  AMO  PC RPORMAMCI  OtJlCTi  vlt  *0*  Tm(  CWlRR|MT 
RATlMO  RfR'OO  TOOK  PUACt  OM _  -  _ 


RATIO  OPPlCIR'S  I  Ml  T  l  A  LA  . 


_ PART  IV  -  RATIO  QPPICIR  «  t  —i—  »»«  pin+m. 

t  «TATt  VQUR  HOMiPtCAMT  OuTif}  AMO  MI^ONUK  L  i  Tif* 


OUtV  TiTLl  »• 


Tm|  PO»r'OM  COOfl  -t 


»  I  MO*  CAT  |  TOUR  MAJOR  PfRPORMAMCI  ORJlCTivft 


OA  67— 6—  1  io'*(ono»  h»  »i'i  osioum 
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t.*t  *0^*  f'CN.*  CANT  CONf«'IOTlONS 


tlONATuMC  AND  OATf 


PART  V  -  RATffl  AMO/ On  )NTC NMf  OlATt  RATfft  .*«.«•  •«*  tnmm-mt  o*  f*i  /V#  ft  «  «ftoi« 
/'WM'V  "fA^»  •«•  r«4M(«4(  »«M  «0«r  ««(•««  04  4  r— # 


•  AATIIl  COUWINTI  iO0h»*mii 


I’ONA  TyWf  AN  0  OaT|  . 

ft  >NT(  AMIDiATf  AATI  H  COMMINTg  (0#n#«w4/  -  -  _  - 


ii"d»*AtwAi  AN  0  O  A  T  C 

DATA  REOUIREQ  ev  THE  PRIVACY  ACT  OF  1874  H  USC  552m,  ~  ~ 


1.  AUTHORITY:  Sm  301  Title  3  USC.  Sec  301J  Title  10  USC 

X  PURPOSE  DA  Form  67  —  $.  Officer  Evaluation  Report.  wt>m  m  the  primary  eource  of  information  for  officer  p*f»onMl 
maria  cement  d«twioni  DA  Form  $7  —  8—1 .  Officer  Evaluation  Support  Form.  • ervaa  **  a  guide  for  th«  rated  officer  •  perform 
met,  development  of  tbe  r«ud  officer,  enhance*  the  iccoaptuhintnt  of  the  organ  nation  mueion.  »nd  rvovide*  additional 
performance  :oforraaUoo  to  tbe  rating  chain 

X  AOUTtNl  USE:  DA  Form  67—8  will  be  maintained  in  the  rated  officer*  official  military  Personnel  Filr  iOMPF  >  *nd 
Carter  Manage  me  ot  Individual  Fil«  iCMIF)  A  copy  will  be  provided  u>  the  rated  officer  either  directly  or  tent  to  the 
forwardiftf  addree*  above  in  Pvt  I.  DA  Form  67—8  Da  Form  $7—  §—  1  *  for  orga  rotational  uee  only  end  will  be  returned  to 
the  rated  officer  after  renew  by  the  rating  chain 

4.  DISCLOSURE:  Declotive  of  the  rated  officer*  S8N  iPvt  1.  DA  Form  67—8)  u  voluntary  However,  failure  to  verify 
the  SAN  moy  retell  in  a  delayed  or  erroneoue  proceaamg  of  the  officer «  OER  Diactoeure  of  the  informetion  in  Part  IV. 

DA  Porm  67—8—1  ia  voluntary  However,  failure  to  provide  the  information  reqeeeted  will  reeult  in  tn  evaluation  of  the 
mted  officer  without  the  beneflta  of  that  officer'*  com  menu  Should  the  reted  officer  uae  the  Privacy  Act  u  e  baeie  not 
to  provide  the  information  requeued  >o  Part  IV,  the  Support  form  will  conuro  the  rated  officer  *  itatemem  to  that  rffect 
and  be  forwarded  through  the  rating  chain  m  accordance  with  AR  $23—105 
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Appendix  C: 


U*MC  HTMUS  ««« (lit* 


ncrnoouccc  at  govcrnmcni  exrt 
ntvNOtiw  not  oi  M*»o*a  lr»  uiiwy  wnokjhow 
UMIO  llv  HIM  HOOVMO}  M  NO  Hurl"  M 


1/UttC  IlfNltt  U*0«T  P*9*  2 
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_ I _ 

OCCAUON 

wo«ti*40  seniors  arnptcATtoN 

1  cortity  that  on  th*  iftmnoi  dot*  shown  in  Horn  3  at  Soction  A.  1  wo*  tho  ••porting  Sontor  lor  only  the**  Marino*  ot  th* 
soma orad*  a* shown  m  Horn  1  Shot  Soction  1  Tho** Marino* oro  ALPHAlfftCALit  USTtfi  bolow.  1  rank  this  Mann* a* 

•4  -  l*nht  ton*  SSnrmwt  nwSM  Outstanding  in  I«e  and  h  mm*  MS  *  not  anwUrnfciwi 

«AMC  ium  Sow  Ml)  | 

!  r«o»  1 

■■Bd 

••Ntan  | 

"•Oft 

a 

•» 

a 

o 

o 

c 

o 

PI 

0 

> 

SlCNATVtf  —  -  -  —  DAT!  -----  Q\ 

_  _  .  .  _  _  O 

tlVieWING  Off  tCtlTS  <TTiytCATK>N 

•  .  □  I  h**o  not  Sod  suMki*ai  opportunity  to  ob**c»*  ttiii  Morin*,  so  I  hoy*  no  commons. 

].  0  I  h*u*  hod  only  limited  opportunity  to  ofctorvo  it**  Morin*,  hut  (torn  whot  I  hovo  ob«*cv*d  I  gonorolly  concur  with  tho 

••porting  Senior*  mortis  In  Homs  I  So  and  b. 

].  0  I  h*y*  had  swHlciont  opportunity  to  ob**rv*  thi*  Morin*,  and  concur  with  tho  ••porting  Seniors  marks  in  Homs  I  So  ond  b. 

4.  □  I  hovo  hod  sufficient  opportunity  to  obsert*  thi*  Morin*,  ond  do  not  concur  with  th*  ••porting  Senior's  morht  in  lt*tns 

IS*  and  b.  I  would  oroluat*  thi*  Morin*  a* _ (H*m  1  Jo)  ond  ronk  this  Morin*  o*  __ _ ol 

_ (only  rank  thot*  *voluot*d  a*  Outltondlng  (OS)). 

REMARKS  {mandatory  H  lt*m  4.  obey.  i*  checked) 


sighatum 


OATI 


NQTf.  Th#  irtformoPto*  WIU  NOT  to  —r  c— pryqrwm. 
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«.  COMMCMTS  II 
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*  ^  "’"C-  U_£T  X.  To  p  ''-fc  r  rt-ie  I 
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—  you. 

tC«r>  plOLC Cr»+* 

^  -  N««d  +©  i^o»'  K  T-**5 

5Hi4crn  **©w»C<±^e  - 

w.fh  p^rsc^nci 

a.nd  rf  vifi  Halion  5j<5*C* 

Lexers  h  1  C  5(C.U3 


’  V0**-  rr'U-Sr  CnVo^cC. 
b  To-ocwxai'S 
I  ouxp  to  -tc  'I  t-u^O  OV 
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Hoj  rca,t>  / 
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a  grta.-t  idea.  -  +rt 
fifiAuicnts  Icveo  i  t 
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touj-  5+vU Jeoi-S  «u.+  <f [£<j 
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--  tt«>  noAt  time  itict  to 
-Wi£  i  nS+rivC  tO>-5  ^u.'dC 

.4A.-hcovoJ  5c,  U 


Af  form  724.  AUG  M 


PERFORMANCE  FEEDBACK  WORKSHEET 


rrMNOTM*.  tuwimo  ooau.  a  no  *oo<Tio«Ak  co—untt 

Oggaj^iAATiQAJAt,  se.u-t-i  £ccnt) 

Your  ©Hidenh  evcUs  are.  UAuallij  •fUrne-d  ir\  the  they  are-  <du.C  Pl«-r| 

ahead  and  get-  thens  ,*  ea»rtg  so  wlC  have.  mere,  options  m  the- 
Circa-  o-f-  -Student"  nnanogernent" 

Pudgmeu-st-  <Xnck  fceeiSlonS 

-  As  -treun.n^  mi5S<©n  Cornruander  i|Ou_  rrxxjec.  goad  decisions,  Contmujn^ 
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—  You-  ne© d  ho  re.la.x-  when  students  mterupt  tfCu.r  planned  presentation 
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Cormnarncahion  SK.1H3 
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Appendix  F: 


I.  NATH  IfANTIPICATION  DATA  iKttd  Aft  )*I0  etnftUf  tofu*  /UU*t  ta  mr  turn/ 


1.  M.i  HI  tUm,  /Wt,  MidAlt  lnutiil 

NTOIWIATHER, 


A  eSSlOOOe  AlPOAT 

item:  14  lor  88  I  **■» 


l  OftOAftOATtON.  COM^Uft  LOCATION 

432d  Aircraft  Generation  Squadron  (p 

A  *UC«M 


It  UNIT  MISSION  OSSCAlPTION 

Responsible  foe  all  on-sguipm«it  maintenance  for  2  combac-r aady  24  primary  assigned  aircraft 
(PAA)  P-14  squadrons.  The  ACS  supports  m-plsce  and  deployed  contingency  operttions  in  tha 
Pacific  theater.  Peacetime  activitiaa  include  supporting  daily  flying  training  at  home 
station  and  various 


ill. JOS OtKftiPTiON  vouTrntU:  off  icer-in-CMcge,  14th  Aircraft  Maintenance  Unit  (ANU) 

a  car  oi/Tits.  tasks,  amo aaaMsisiaitinaa:  provides  on-equip*«»t  maintenance  for  24  P-14C/D  aircraft 

in  ths  air-to-air  and  air-to-ground  missions.  Responsible  for  2SS  personnel  In  14  APSCs, 
performing  maintenance,  launch,  recovery,  and  weapons  loading  functions.  Acts  as  senior 
maintenance  officer  during  deployments.  Also  responsible  for  training  and  training 
records  for  all  unit  personnel  and  S2.SN  worth  of  unit-owned  support  equipment  as  well  as 
aerospace  ground  equipment,  vehicles,  and  AMD  facilities.  SIGNIFICANT  ADDITIONAL  DUTIES: 
Squadron  Mobility  Officer.  _  _ _ _ 


iv  impact  on  mission  accompushmint 

-  Led  AMD  in  COPE  THUNDER  88-3 

—  2(8  successful  training  sorties  without  maintenance  air  abort  while  operating  fron 
deployed  location 

•  Developed  new  flightline  mobility  procedures 

—  24  combat-ready  aircraft/28  inersaents  of  error-free  cargo 

-  Achieved  85  percent  systems  effectiveness  by  integrating  all  electronic  combat  subsystems 
into  a  comprehensive  electronic  combat  program 

—  75  percent  is  MAJCOn  standard 

-  Developed  coecutenzed  maintenance  system  procedures  2  wssfcs  aarly _ 
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1  )*h  Kn— liege 

knowWdgr  ttdva'rd  lo  period"  dut-er  *«e<l'»i'Y 
Slrwei  to  mprO*!  INM  knowledge 


2  InSinNi  SAiin 

Sen  and  i*v<xm  »ta«da»di  womt  otnin 

lortm  teamwork  O\0*Ayt  rt.tiatnm  Srd^onidtrtt 


1.  VrofeitlenAl  Owalitiei 

(an-bni  loyalty.  dni'ptvw  d*d<at«rv  Ttegrrir  *"d  normt? 
AOM>«t  to  A»  »or»»  ttandardt  Aktrctt  pmo nH  rrteonwOAtr 
t  law  and  odiertwe 


a.  OrgeoiaAttexel  PkliH 

piim.  (oorowuttt.  MMCuiek.  and  na  mounts  tNrtttmM 
Meett  twkoemei 


S  Judgment  to*  Oocrtiona 

Maker  twnetr  and  act  urate  deoexxx  (mpNetxn  tog<  it 
lew  ntatrg  e«tawH  tomeoture  et  ti>«tt'v4  vtuatem 
tKogtkWi  opoortwnrtwr  ieuwet  mevmal  tvprw <o» 


□ 


□ 


□ 


□ 


□ 


4.  Cewummkstie««  saiih 
ottem.  tmeaAi.  and  «»met  e*tect'»e*t 
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VI.  RAT4A  OVtAALU  tmUMINT 

Capt  nertiweatie:  •$  l«d»:ihip  had  *  positive  impact  3n  the  entire  squadron.  He  took  this 
low-performing  AMU  and  brought  most  maintenance  indicators  up  to  or  abov«  coamand  standards. 
His  diligence  1:  loved  completion  of  (out  urgent  action  TCTOs  while  preparing  aircraft  for 
COPC  THUNDER  98-3.  His  AMU  supported  a  il  JTE  rate  in  spite  of  malor  aircraft  structural 
restrictions.  He  danonstrated  potential  for  increased  responsibility  in  his  daily  integration 
of  the  management,  technical,  and  adeimstrative  facets  r^ulred  of  a  successful  AMU. 

Capt  Her r iweather  is  an  outstanding  pette'mer.  Send  him  to  intermediate  service  school  in 
reeidence. 


Serfeemawe  feedback  Mctxmoii«h«d  continent  with  me  direction  in  Ar  '  IS- '  w  Ilf  not  acse mp/iehed.  am  tfo  rmatn.l 


NAmT  OMAOS.  M  O*  SVC.  ones  coeeo.  VOCATION 

MAUXWE  B.  ERVIN,  Lt  Col,  USAT 
432d  Aircraft  Generation  Sq  (PACAF) 
Hisawa  AR.  Japan _ 


OUTV  tiki 

^niwiwnd.r 


U2d A 


xxx-»x-xxxx 


rrr A.f r  fji»rttinii  <lq 


JlOMATViat  U 

ft.  C'Urw\ 


"Sari 

.  2fl_fan  aa. 


VII.  AOOlTlONAL  RATCS  OVCAALl  ASStSSMCNT  •  CONCUR®  NONCONCUR 

A  dedicated  maintenance  officer  with  aicellent  potential.  Capt  Mernweather  providee  strong 
leadership  to  his  W1U.  His  abilities  resulted  in  his  AMU  being  selected  as  the  432d  Tactical 
Fighter  Ming  Aircraft  Maintenance  Unit  of  the  Quarter  for  the  fust  time  this  year.  This  is 
indicstive  of  the  depth  of  his  Apeuence  and  knowledge.  He  is  an  outatanding  maintenance 
officer  and  a  valued  manner  of  the  wing. 


hums,  goads  I*  o»  tve  onon  cowo  vocation 
STEVEN  A.  BURT-ESON,  Col,  USAF 
432d  Tactical  Fighter  Ming  (PACAF) 
A8 uL.J«*n - 


OUT V  TlTl* 

Deputy  Commander  /or 


UN 


Maintenance 


|OAT| 

1  3  tut  ,89 


VIII  RIVlCWCR 


concurH 


NONCONCUR  HI 


NAMC  OOAOC  e«  or  TVC  0«CN  COWO  VOCATION 

ANDRW  D.  JONES,  Col,  USAF 

432d  Tactical  Fighter  Ming,  (PACAF) 

Hisawa  AB.  Japan _ 


OUT V  TltVf 

Ming  Commander 


SSN 

xxx-xx-xxxx 


tlCNATual 

/MdvuJ  /0C^OU_J 


OATI 

i  Mac  99 


•AMfvci»om 

All  R*co/TUT3*n<jji.Qm  moil  M  bmd  on  oarformanca  «n<j  ?hf  oottnt.*i  o«ft*o  on  tfvrt  performance  ^'omoi.on  'KommfnQii ior*t  ir« 
prohibited  Oo  -»ot  o r  comm#<n  on  cWDpirt-oo  o*  or  t  nroilment  m  advanced  education  ^tviowi  o r  anticipated  promotion 

recommendation*  on  Af  t  j/m  709  0€R  ."doraemant  'twit,  family  activity  marital  ftatut  race  m  athn-e  ot^n.  tft  or  rtiyo n 

Ratae:  *ocu»  y«jr  evaluation  in  Section  IV  on  what  the  officer  6r6  hour  wei'  hf  or  af*  drf  >t  and  ho*»  the  officer  conmouted  to  mmion 
aeppm  oi  i  iftmtri  t  Wr  iw  n  conote  buN«t  '  format  Your  com  mam  I  ift  Section  VI  may  >nciyd«  recommendation*  for  augmentation  or 

auifnmam 

Additional  Rato#  Certfuiiy  '**>+*  the  .'ittr  »  evaluation  to  #n*ura  >t  i  accurate  unb«a*ad  and  umnfijted  if  you  d-ta^te  you  may  as* 
the  ratff  to  rfw.rw  hi»  or  h#»  evaluation  You  may  not  direct  a  c^an^a  r%  tha  »y#iu»tion.  if  you  it.n  d  uyrN  s*nn  th*  '|<tr  mar%  NON 
CONCUR"  and  eaO'em  y<kj  may  '<xiu4a  'ocommaryjatiom  for  au^mantation  o*  aaa*9nmam 

Rayimaa*  Car«fuiiy  raviow  tha  r*tar  »  and  additional  rrttn  'lliNl  and  commantt  if  that*  av*uei>on«  a*  acrurata  unO*at«d  and  *mn- 
Hat  ad.  rna^t  tha  form  CONCUR"  and  n9n  th#  form.  If  you  duafraa  «mh  orav«oua  evasion  you  may  iu  trom  to  raviow  tha*r  avaiua- 
t*on*  You  may  not  diraci  thorn  to  change  tha»r  aopraitaif.  If  you  ittil  diu^aa  mth  tha  additional  r*i#r  mart  NONCONCUR"  and 
e>  plain  m  Soction  VMI  Op  not  gia  NONCONCUR"  umpty  to  provtfa  com  man  ti  on  th#  rtoort 
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Appendix  G: 


I.  HATH  IQ1NTIT  ICATlQN  QATA  aF*  lt-10  f,U, 


1.  KIM  Lilt  Fmt.  W|44J<  Mill*,  I  MM  1  b*AO«  «  0»*»C 

HARRIS,  JOHN  T.  XXX -XX- XXX X  C*Dtiin  263$ 


A  omamiiaTion  co— m.  AMO  v. oca  r  i cm  t  Ml  COOK 

ctmenc  of  Phytice,  Oman  of  Faculty  (USAfA) ■  Colorado  Spring*,  Colorado  US08PBCC 


II.  UNIT  MISSION  OUCAimON 

The  Physics  Department  it  on*  of  19  academe  department*.  Department  teechee  3  introductory 
physics  coursst  to  2991  cadats  annually  and  2S  advanced  physics  coursss  to  93  physics  M3ors. 
Conducts  basic  spacs  physics,  lassr  and  astronony  research  for  savsral  operational  comands. 
Providas  officst  rols  models  and  carssr  countalinq  to  cadsts. 


ill.  jo«  Of scsirriON  i.  ourvrm.*  Courts  Dirsctor  and  instructor  of  Physics 
s  kit  Burns,  tasks  ssspei—sitmts-  Supervisee  II  instructors  who  annually  tsach  25  tactions  of 
introductory  calculus-bassd  physics  tc  1211  cadsts.  Develop*  curricula,  Mtablishss 
oducational  standards,  and  determine*  9radst.  instructor  fcr  42  cadsts  m  aubjact  araas  of 
classical  mechanics,  thermodynamics,  and  fluid  mechsnici.  Conducts  classas,  motivates,  and 
count  a  la  cadats.  Conducts  basic  optica  rasaarch  funded  by  tl  s  Air  Porca  weapons  laboratory. 


IV  PROMOTION  MCOMMINOATION 

Capt  Harris  is  ona  of  our  top  company  grade  instructors  meeting  this  board.  His  superior 
record  of  performance— young  eat  Hinutas»an  craw  contnander  at  Hinot  a/8,  ground-breaking 
rasaarch  at  the  Air  Porca  Weapons  laboratory— bring  a  raal-lifa  relevance  to  his  courses.  As 
a  result  of  his  rasaarch  proposals,  ha  acquired  S13IR  worth  of  laser  equipment  and  organized 
a  research  team  of  5  officars.  Ha  is  now  the  principle  investigator  for  this  project  which 
has  significant  SDt  application.  His  unique  bland  of  operational  experimce,  technical 
expertise,  and  supervisory  talent  mark  him  for  a  bright  future.  Promote  to  ma}or  this  board 
and  select  for  intermediate  service  school. 


V  PROMOTION  IONS 


"□  '“"I  I 


IX.  OVlAALl  XICOMMtNOATlON 


vi  group sui 

VII  »OANO 

NA 

XX  XXX 

VIII  SfNlOA  XATIX  to 


O©  NOT 


□ 

□ 


x  JiNioa  kATia 

mams  qaaoi  ea  or  tvc  o*on  COMO  location 
rtARTlN  W.  DALEY,  Col,  USAP 
OS  Air  Poree  Acadtmy  (USAP A) 

Oolotado  Springs,  Colorado _ 

OUTv  T|?l| 

Permanent  Professor  and  Heed,  Department  of  Physics 


xxx-xx-xxxx 


Instructions 

Review*  previous  OERs.  OPRs.  Education/Training  Reports,  and  Supplemental  Evaluation  Sheets  Discuss,  if 
needed,  the  officer's  performance  v*ith  officials  m  the  supervisory  cham  Evaluate  the  officer  s 
performance  and  assess  his  or  her  potential  based  on  performance  Oo  not  consider  or  comment  on 
enrollment  m  or  completion  of  professional  military  education  or  advanced  academic  education 

Provide  an  accurate,  unbiased  assessment  free  from  consideration  of  race  sei  ethnic  ongm  age  religion 
or  marital  status. 

Provide  the  officer  a  copy  of  this  report  approximately  30  days  prior  to  the  board  for  which  this  report  is 

prepared. 
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SURVEY 

OF 

OFFICER  PERCEPTIONS 
OF  THE 

OFFICER  EVALUATION  SYSTEM 


USAF  SCN:  89-42,  expires  1  Aug  89 
POC:  AFIT/LSG  (Cipt  Hale) 
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The  purpose  of  this  questionnaire  is  to  obtain  data  concerning  your  perceptions  of 
the  Officer  Evaluation  System.  These  data  will  be  used  by  the  Air  Force  Institute  of 
Technology  to  gauge  officer  acceptance  of  the  recently  revised  system. 

Please  be  assured  that  all  information  you  provide  will  be  held  in  the  strictest 
confidence.  The  goal  is  not  to  identify  individual  responses,  rather  to  identify 
differences  of  perceptions  within  categories  or  groups  of  officers,  for  example 
company  grade  versus  field  grade.  When  the  results  of  the  study  are  published 
readers  will  not  be  able  to  identify  specific  individuals. 

KEYWORDS 

The  following  terms  or  acronyms  will  be  used  throughout  the  questionnaire: 

1.  OER  system:  The  Officer  Effectiveness  Reporting  system  used  from 
1978-1988. 

2.  OER:  Officer  Effectiveness  Report  (AF  707),  used  to  provide  a  long-term 
cummulative  record  of  officer  performance  from  1978- 1988. 

3.  OES  :  The  current  Officer  Evaluation  System  (as  of  1  Aug  1988), 
composed  of  the  Performance  Feedback  Worksheet,  the  Officer 
Performance  Report,  and  the  Promotion  Recommendation  Form. 

4.  PFW:  Performance  Feedback  Worksheet  (AF  724),  a  hand-written  form 
used  to  facilitate  communication  during  ratee/rater  feedback  sessions. 

5.  OPR:  Officer  Performance  Report  (AF  707A  or  707B),  used  to  provide  a 
long-term  cummulative  record  of  officer  performance. 

6.  PRF:  Performance  Recommendation  Form  (AF  709),  used  to 
communicate  a  promotion  recommendation  from  the  senior  rater  to  the 
central  promotion  board. 

7.  Ratee:  The  individual  officer  being  rated  by  his  or  her  supervisor. 

8.  Rater:  The  individual  officer  who  rates  his  or  her  subordinate. 

INSTRUCTIONS 

Enclosed  is  a  blue  Optical-Scanning  form  (AFIT  HE)  that  provides  seven  choices 
(1-7)  for  each  item.  If  the  AFIT  11E  was  lost  or  is  damaged,  please  write  your 
answers  on  the  questionnaire  and  return  it  instead  of  the  blue  form. 

The  attached  questionnaire  contains  49  items  (individual  "questions ")  and 
should  take  no  more  than  25  minutes  to  complete.  All  officers  should  answer  items 
1-34  in  sections  1  and  2,  then  items  35-49  based  upon  specific  instructions  provided 
before  each  of  the  remaining  sections.  If,  for  any  item,  you  do  not  find  a  response 
that  fits  your  situation  exactly,  mark  the  one  that  is  closest  to  the  way  you  feel. 

Please  use  a  #2  pencil  and  completely  fill  in  only  one  of  the  numbered  circles 
(1-7)  that  corresponds  to  your  answer  for  each  item.  Completely  erase  any  errors  or 
stray  marks. 

When  you  have  finished,  place  only  the  blue  AFIT  HE  in  the  return  envelope 
provided  and  put  in  distribution. 
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OES  QUESTIONNAIRE 


The  following  questionnaire  is  concerned  with  your  perceptions  of  the 
current  Officer  Evaluation  System  (OES).  Read  each  item  carefully 
and  choose  the  alternative  that  more  accurately  describes  your 
present  situation. 

Section  1:  Using  a  pencil,  darken  the  entire  circle  on  the  data  entry 
form  that  corresponds  to  your  response  for  each  item.  Erase  all 
errors  completely. 


What  is  your  current  rank? 

Lieutenant  Colonel 
Colonel 
Other 

4.  Major 

What  is  your  gender? 

1.  Female 

2.  Male 


1.  Second  Lieutenant  5. 

2.  First  Lieutenant  6. 

3.  Captain  7. 


From  items  3  and  4  choose 
major  command. 


only  one  answer  that  identifies  your 


3.  To  what  major  command  or  headquarters  are  you  currently 
assigned? 

1.  AFLC  4.  ATC 

2.  SPACECMD  5.  MAC 

3.  AFSC 

4.  To  what  major  command  or  headquarters  are  you  currently 
assigned? 

1.  SAC 

2.  Tactical  Air  Forces  (  PACAF,  TAC,  USAFE  ) 

3.  HQUSAF 

4.  Other 
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5.  To  which  organizational  level  are  you  assigned? 


1. 

Below  MAJCOM  level 

2. 

MAJCOM  level 

3. 

Above  MAJCOM  level 

6. 

What 

is  your  current  aeronautical  rating? 

1. 

No  aeronautical  rating 

2. 

Navigator 

3. 

Pilot 

4. 

Other  aeronautical  rating 

7. 

What 

is  your  racial  or  ethnic  background? 

1. 

American  Indian 

2. 

Black/Black  American/Afro-American 

3. 

Caucasian/White  (Other  than  Spanish  Speaking) 

4. 

Oriental/Oriental  American  (Asian  American,  Chinese, 
Filipino,  Japanese,  Korean) 

5. 

Spanish  speaking  origin  (Chicano,  Cuban,  Latin 
American,  Mexican) 

6. 

Other 

8. 

How  i 

many  veers  of  commissioned  service  have  vou  completed? 

1.  Less  than  one  year  4.  11-15  years 

2.  1-5  years  5.  16-20  years 

3.  6-10  years  6.  21  or  more  years 

9.  How  many  years  of  total  active  federal  military  service 
(TAFMS)  have  you  completed? 


1. 

Less  than  one  year 

4. 

11-15  years 

2. 

1-5  years 

5. 

16-  20  years 

3. 

6-10  years  or  more 

6. 

21  or  more  years 

On  your  last  PER  under  the  previous  system  (prior  to  1  Aug  88) . 

10.  Did  you  receive  well  above  standard  (fire-walled)  ratings  for  all 
ten  front-side  performance  factors? 
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1.  Yes 

2.  No 

3.  I  was  never  evaluated  under  the  previous  OER  system 

11.  Did  you  receive  all  top  block  ratings  (from  rater,  additional 
rater,  and  endorser)  on  the  back-side  Evaluation  Of  Potential 
rating  scale? 

1.  Yes 

2.  No 

3.  I  was  never  evaluated  under  the  previous  OER  system. 

12.  What  endorsement  level  did  you  receive?  (use  equivalent 
civilian  rank  if  applicable) 


1. 

Colonel  or  below 

4. 

Lieutenant  General 

2. 

Brigidier  General 

5. 

General  or  higher 

3. 

Major  General 

6. 

Not  evaluated  under  the 

OER  system. 

Under  the  current  Officer  Evaluation  System . . 

13.  What  did  vou  receive  for  your  most  recent  Promotion 
Recommendation? 

1.  Definitely  Promote 

2.  Promote 

3.  Do  Not  Promote  This  Board 

4.  I  have  not  been  evaluated  for  promotion  under  the  new 
system. 

14.  What  do  vou  expect  to  receive  for  your  next  Promotion 
Recommendation? 

1.  Definitely  Promote 

2.  Promote 

3.  Do  Not  Promote  This  Board 

4.  I  will  not  be  evaluated  under  the  new  system  (retiring, 
resigning,  etc.) 
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Please  use  the  Agree-Disagree  scale  shown  below  to  answer  each  of 
the  items  in  Sections  2  through  6.  Using  a  pencil,  darken  the  entire 
circle  on  the  data  entry  form  that  corresponds  to  your  response  for 

each  item.  Completely  erase  all  errors. 
****************************************************************** 

Neither 

Strongly  Slightly  Agree  nor  Slightly  Strongly 

Disagree  Disagree  Disagree  Disagree  Agree  Agree  Agree 
I  I  I  I  I  I  I 

1  2  3  4  5  6  7 


Section  2:  Answer  items  15-34  even  if  you  have  not  been  formally 
evaluated. 


15.  The  Officer  Evaluation  System  (OES)  provides  lieutenants  and 
captains  with  accurate  feedback  on  their  duty  performance. 

16.  The  OES  provides  me  with  an  accurate  assessment  of  my  duty 
performance  and  potential  based  upon  that  performance. 

17.  OES  forms  and  other  documents  in  promotion  folders  permit 
promotion  board  members  to  select  the  best  qualified  officers 
for  advancement. 

18.  An  officer  whose  duty  performance  is  well  below  average  is 
easily  identified  under  the  current  OES. 

19.  An  officer  whose  duty  performance  is  below  average  is  easily 
identified  under  the  current  OES. 

20.  An  officer  whose  duty  performance  is  average  is  easily 
identified  under  the  current  OES. 

21.  An  officer  whose  duty  performance  is  above  average  is  easily 
identified  under  the  current  OES. 

22.  An  officer  whose  duty  performance  is  well  above  average 
(below-the-promotion-zone  quality)  is  easily  identified  under 
the  current  OES. 


**********************************+******************************* 

Neither 

Strongly  Slightly  Agree  nor  Slightly  Strongly 

Disagree  Disagree  Disagree  Disagree  Agree  Agree  Agree 
I  I  I  I  I  I  I 

1  2  3  4  5  6  7 

****************************************************************** 

23.  More  time  is  spent  on  Officer  Performance  Report  (OPR) 
appearance  than  content. 

24.  Too  much  time  is  taken  by  the  entire  OPR  process  (from  initial 
draft  to  becoming  a  matter  of  record). 

25.  The  time  needed  to  prepare  OPRs  is  well  worth  the  effort. 

26.  I  should  have  more  input  into  my  OPR  before  it  becomes  a 
matter  of  record. 

27.  My  senior  rater  is  better  qualified  to  determine  my  promotion 
potential  than  a  central  promotion  board. 

28.  An  officer  has  an  equal  chance  of  receiving  a  "definitely 
promote"  recommendation  regardless  of  the  organizational 
level  to  which  he  or  she  is  assigned. 

29.  Promotion  decisions  were  fair  and  just  under  the  previous  OER 
system. 

30.  Promotion  decisions  are  fair  and  just  under  the  current  OES. 

31.  Performance  was  fairly  and  accurately  appraised  under  the 
previous  OER  system. 

32.  Performance  is  fairly  and  accurately  appraised  under  the 
current  OES. 

33.  Under  the  previous  OER  system  officers  received  clear  and 
timely  feedback. 

34.  Under  the  current  OES  officers  receive  clear  and  timely 
feedback. 
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****************************************************************** 


Neither 

Strongly  Slightly  Agree  nor  Slightly  Strongly 

Disagree  Disagree  Disagree  Disagree  Agree  Agree  Agree 
I  I  I  I  1  I  I 

1  2  3  4  5  6  7 

****************************************************************** 


Section  3:  Answer  items  35-44  only  if  you  have  received  feedback  from 
your  rater  using  the  Performance  Feedback  Worksheet  (PFW).  If  not, 
skip  to  Section  4. 


During  my  latest  feedback  session  using  the  Performance  Feedback 
Worksheet . 

35.  Specific  examples  of  my  performance  were  cited. 

36.  The  comments  concerning  my  performance  were  objective. 

37.  There  was  two-way  communication  between  my  rater  and  me. 

38.  Both  my  strengths  and  weaknesses  were  discussed. 

39.  My  rater  listened  to  what  I  had  to  say  in  response  to  his 
feedback. 

40.  The  focus  was  on  duty  performance  and  officership. 

41.  The  examples  of  my  behavior  cited  by  the  rater  were  drawn 
from  the  entire  rating  period,  not  just  recent  events. 

42.  My  rater  and  I  discussed  a  course  of  action  for  improved 
performance  for  my  next  feedback  session. 

As  a  result  of  the  latest  feedback  session . 

43.  I  have  a  better  understanding  of  what  is  expected  of  me. 

44.  I  will  be  able  to  do  my  job  better. 
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****************************************************************** 


Neither 

Strongly 

Slightly 

Agree  nor 

Slightly 

Strongly 

Disagree  Disagree  Disagree 

i  i  i 

Disagree 

i 

Agree 

i 

Agree 

I 

Agree 

I 

1 

i  1 

2  3 

• 

4 

t 

5 

1 

6 

7 

****************************************************************** 


Section  4:  Answer  items  45-47  only  if  you  have  provided  feedback  to  a 
subordinate  using  the  Performance  Feedback  Worksheet  (PFW).  If 
not,  skip  to  Section  5. 


45.  The  format  of  the  PFW  makes  it  easy  to  complete. 

46.  The  rating  scales  are  useful  in  portraying  how  much 
improvement  is  needed  in  an  officer's  performance. 

47.  The  PFW  is  a  valuable  tool  for  conducting  a  well  organized 
feedback  session. 


Sectiop  5:  Answer  items  48  and  49  only  if  you  have  completed  an 
Officer  Performance  Report  (OPR)  as  a  rater. 


48.  The  OPR  is  easier  to  complete  than  its  predecessor,  the  OER. 

49.  The  OPR  is  a  better  tool  for  documenting  job  performance  than 
the  OER. 


THANK  YOU 

Please  place  your  answer  sheet  in  the  envelope  provided  and  put  it  in 
distribution. 
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